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(57) Abstract: An apparatus and method is provided to allow an operator of a vehicle to have an early warning of traffic conditions 
that may effect his or her travel, traffic conditions in the immediate vicinity of the operator's vehicle and traffic conditions that 
are a litUe bit further than the immediate vicinity of the operator's vehicle but still of significance to the operator at the time An 
electromc vehicle safety early warning system includes a receiver which receives content signals representative of surrounding traffic 
condiuons, a processing subsystem which processes the signals to format the content signals for display, and a display apparatus 
which provides a visual display presentation of the surrounding traffic conditions. 



)NSDOCI0:<WO 0143104A1 I > 



WO 01/43104 



PCT/USOO/33433 



-1- 

METHODOLOGY, APPARATUS, AND SYSTEM FOR ELECTRONIC 
VISUALIZATION OF TRAFFIC CONDITIONS 

RELATED APPLICATIONS 
Not Applicable 

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
S Not Applicable. 

BACKGROUND OF THE INVENTION: 

This invention relates to a methodology, apparatus, and system for electronic 
visualization of traffic conditions and in-vehicle as well as extemal-to-vehicle-based 

10 subsystems and networked combinations thereof. 

Users travel often in private vehicles such as cars (buses, trucks, vans, etc.) or 
boats which are confined to certain rights-of-way, namely roads or waterways. The 
nature of these rights-of-ways are that the individual vehicles tend to accumulate in 
patterns comprising traffic, where one present vehicle is said to be following a vehicle 

IS that's in front of it; likewise there are vehicles behind the present vehicle. Even though 
each vehicle may have its own individual destination, tihe path from an origination point 
to the destination point often involves many vehicles travelling over substantially the 
same right of way. A common example of this would be commuters driving into work 
via expressways or other highways, where they join the highway at some point close to 

20 their origination point and exit the highway at some point close to their destination, 
meanwtule sharing the highway with a plurality of other vehicles also going in the same 
general direction. 

As any commuter is familiar with, the collection of vehicles - especially during 
peak travel times - becomes a difficult traffic problem to navigate. Often the operator of 
25 a vehicle finds himself or herself stuck immediately behind a stopped vehicle in a solid 
mass of traffic without knowing what the circumstances are that caused the traffic to be 
stopped and whether there is any action that the operator can take that would improve 
that operator's ability to get to their destination in a timely manner. 
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Prior art solutions generally involved "rubbernecking", which is the process of the 
operator trying to see outside of his or her vehicle in order to determine which lane is the 
optimal lane to be in; listening to radio traffic reports, which are relayed from mass media 
concerns such as news radio stations; and alternatively using the newer mobile or cellular 

S phones to call an information traffic service. 

The problem with the prior art is that, in the case of the operator simply trying 
to see what is around his or her vehicle, the scope of visibility is too limited. The 
operator can only see a short distance in any direction from his or her vehicle and that 
line of sight may be blocked by other vehicles that happen to be in the way, or 

10 structures, or bends of the road preventing the operator from actually getting any useful 
information out of looking ahead. 

The problems with mass media and traffic services are the opposite. The ability 
to look anywhere within a road system is certainly available, but the information - 
because it has to be presented to a mass market is not detailed enough for a given vehicle 

15 operator. For example, a traffic report might indicate that a particular highway is busy 
from point A to point B. Often that traffic report information is delayed by the amount 
of time that it takes the report to arrive at the news service, be properly added to their 
transmission program, and subsequently relayed to the operators of the vehicle, during 
which time the road situation may have changed. Furthermore the vehicle operator may 

20 only be interested in their ability to get to point C where reported points A & B are 
irrelevant. 

Another problem with the prior art is that if an operator is unfamiliar with the 
traffic in a given area, for instance a new visitor or professional driver that has just 
arrived in a city, he or she may not be familiar with the local colloquialisms and common 

25 names for roadways that otherwise appear^n an atlas as route numbers. For example, a 
visitor to Washington, D.C. might have difficulty knowing which road is the Beltway if 
they have never been to Washington before. Similariy, in the Bay area of California, 
finding the Almaden expressway, or in the Chicago area, finding the Eisenhower 
expressway, can be difficult for operators of vehicles that have never been to those 

30 particular locations before, and invariably traffic reports are given in terms of the 
common names for the rights of way. 
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Due to the limited market, these kinds of infomiation are not available in certain 
areas that might be more rural or not serviced by a large news organization, or rights of 
way that are not as commonly traversed such as waterways. As a result, the existing 
solutions for the problem of informing operators of road conditions that may effect their 
5 travel fall short of the ideal of giving the operator the amount of detail of information and 
the amount of resolution of information that would result in the operator having a better 
operating experience and getting to his or her destination in a more efficient manner. 

SUMMARY OF THE INVENTION: 

10 In accordance with the present invention, an apparatus and method is provided to 

allow a operator of a vehicle to have an early warning of traffic conditions that may effect 
his or her travel, traffic conditions in the immediate vicinity of the operator's vehicle and 
traffic conditions that are a little bit further than the immediate vicinity of the operator's 
vehicle but still of significance to the operator at the time. 

15 It is another object of the present invention to convey detailed information about 

very localized traffic conditions to persons other than operators of vehicles, such as 
emergency vehicle crews, or persons involved in the collecting of information about 
traffic conditions for relay, for example to the news stations previously mentioned. 

It is another object of the invention to provide an apparatus and a method that 

20 allows traffic conditions in a defined (e.g., forward) direction of travel of the user, or 
ideally in any direction of travel from an operator's vehicle, to be relayed to the user(s), 
such as from other vehicles, or possibly from fixed or stationary points located at some 
distance from the operator's vehicle or from the user. This relay process can be direct 
from the source of the images, or data about traffic conditions, or it may be relayed 

25 through a series of steps. ^ 

These and other aspects and attributes of the present invention will be discussed 
with reference to the following drawings and accompanying specification. 

BRIEF DESCRIPTION OF THE DRAWINGS: 
30 FIG. 1 is a system diagram of one embodiment in accordance with the present 

invention; 
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FIG. 2 is a structural diagram of an image sensor in accordance with the present 
invention; 

FIG, 3 is a structural drawing of an image sensor wherein camera body 300 
comprises image sensor 310 and lens 320 and an infrared filter 325; 
5 FIG. 4 shows another embodiment of the invention indicating the use of a vehicle 

speed sensor input to control the retractable support; 

FIG. 5 shows a detailed view of an alternate embodiment of the pan and tilt 
mechanism of the present invention; 

FIG. 6 is a schematic diagram of one embodiment of the present invention in 
10 operation; 

FIG. 7 is a schematic drawing of another embodiment of the present invention in 
operation, where road hazard 700 exists on a thoroughfare ahead of a first vehicle 745, a 
second vehicle 765 and a third vehicle 775; 

FIG. 8 is an illustration of an example of a display provided in accordance with 
15 the present invention, illustrating multiple images from ahead in the traffic flow and 
operator controls; 

FIG. 9 is a detailed schematic illustration of a retractable support as in one 
embodiment of the present invention; 

FIG. 10 is an illustration showing additional detail of the mechanism of FIG. 9 
20 wherein the retractable support comprises a retractable support housing 1040 that emits 
a retractable support column 1040, upon which is mounted a pan and tilt means 1020, 
which is in turn affixed to a image sensing means 1000 having an angular field of view 
1010; 

FIG. 11 is an even more detailed view of the image sensing means and the pan and 
25 tilt means of FIG. 10; ^ 

FIG. 12 is a schematic illustration showing one embodiment of the present 
invention utilizing a speed sensor input similar to the one shown in FIG. 4; 

FIG. 13 shows an alternate embodiment of the speed sensing retractable support 
controller means similar to those shown in FIGS. 12 and 4; 
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FIG. 14 shows an alternate embodiment of the retractable support wherein the 
retractable support 1420 variably positions the image sensing means 1410 off to the side 
of vehicle 1400; 

FIG. 15 shows one embodiment of the present invention wherein the retractable 
5 support structure is activated by means of hydraulic piston; 

FIG. 16 shows a schematic drawing of a multi-sensor apparatus as in one 
embodiment of the present invention wherein there are a plurality of image sensing means 
1610, 1615, and 1620, and other sensing means 1625 coupled to processor 1630; 

FIG. 17 shows an embodiment of the present invention as utilized on watercraft; 
10 FIG, 1 8 shows details of one implementation of the mounting means as in FIG. 

17; 

FIG. 19 shows one embodiment of the present invention wherein the system 
comprises a receiver 1970 coupled to a processing means 1930 which is in turn coupled 
to a display means 1940 within the vehicle 1900; 
15 FIG. 20 shows an alternate embodiment of the present invention wherein the 

system comprises an antenna 2060 coupled to receiving means 2070, which in turn is 
coupled to processing means 2030 which is, in turn, coupled to a display means 2040; 

FIG. 21 shows another embodiment of the present invention as used in a central 

station; 

20 FIG. 22 shows an embodiment of the present invention wherein the image sensing 

means 2210 is affixed to vehicle 2200 via the fixed support 2220; 

FIG. 23 shows another embodiment of the present invention, wherein the image 
sensing means 2310 may be operatively elevated above the position of vehicle 2300 via 
the support element 2320; 

25 FIG. 24 shows a schematic of one multiple vehicle embodiment of the present 

invention; 

FIG. 25 illustrates an alternate embodiment of the first vehicle as shown in FIG. 
24, wherein the combination image sensing means and transmission means 2590 is affixed 
to vehicle 2500 via support 2520; 
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FIG. 26 shows an alternate embodiment of the present invention wherein 
emergency vehicle 2600 is equipped with an image sensing and transmission apparatus 
2690 which provides a transmitted signal 2695; 

FIG. 27 shows an alternate embodiment of the present invention wherein a 
5 number of image sensing means 2710, 2720, 2730 are arranged at strategic locations along 
a roadway 2700; 

FIG. 28 illustrates schematically the central station 2805 comprising a variety of 
sources of information including textual information 2800 and visual information 2810; 

FIG. 29 is an example embodiment of the present invention, wherein an image 
10 sensing means 2910 is affixed via support structure 2920 to a road sign 2930 located 
adjacent the right-of-way 2940; 

FIGS. 30A and 30B are representative examples of the types of display 
information that may be presented in accordance with one embodiment of the present 
invention; 

15 FIG. 3 lA is a schematic representation of an alternate embodiment of the present 

invention wherein a combination image sensing and transmission apparatus 3110 is 

located adjacent the roadway 3170; 

FIG. 3 IB is an alternate embodiment to that depicted in FIG. 31 A wherein the 

image sensing means 3115 is directly coupled to a processing means 3145 which is in 
20 turn directly coupled to billboard 3150 thereby providing for integrated information 

display responsive to the image detected by image sensing |neans 3115 of the roadway 

3180; 

FIG. 32 shows one embodiment of the signal receiving and processing element of 
the present invention; 

25 FIG. 33 represents a simplified block diagram of the receiving and processing 

systems of the present invention; 

FIG. 34 shows one embodiment of the present invention illustrating detail of the 
image processing and formatting means 3350 as shown in FIGS. 32 and 33; 

FIG. 35 is a schematic representation of a multiple transmitter embodiment of the 
30 present invention wherein the user's vehicle 3500 receives a plurality of signals via 
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antenna 3510 from sources including a first transmitter 3520 and a second transmitter 
3530; 

FIG. 36 shows a schematic illustration of the operation of an embodiment of the 
present invention wherein a building 3630 blocks the view of roadway 3640 some 
5 distance from the user's vehicle 3600; 

FIG. 37 shows a schematic representation of the one embodiment of the present 
invention as used in watercraft; 

FIG. 38A is a schematic representation of traffic congestion involving an 
emergency vehicle 3800, the user*s view of which is blocked; 
10 FIG. 38B shows a representation of one possible display as shown by display 

means 3850 of the present invention wherein the information received by image sensing 
means 3830 is directly displayed for the user; 

FIG. 38C shows a different display of the present invention that would be 
presented to the user on display means 3850 which comprises a schematic view derived 
15 from information received from image sensing means 3830 and processed to produce the 
overhead schematic view and other user information as shown in the illustration; 

FIGS. 39A and 39B represent an alternate embodiment of the system as depicted 
in FIGS. 38A, 38B, and 38C wherein the emergency vehicle 3800 is again blocked from 
direct view of the user in vehicle 3810 by intermediate vehicle 3820; 
20 FIG. 40 is another example embodiment of the present invention wherein a first 

vehicle 4000 comprises a support means 4010, image sensii^g means 4020, and a 
transmission means 4030; 

FIG. 41 shows one embodiment of the present invention which utilizes multiple 
display means, 4110, 4120, and 4130, all coupled to processing means 4140; 
25 FIG. 42 shows one embodiment of the present invention that supports a relay 

mode of operation; 

FIGS. 43 A and 43B show alternate embodiments of the relay mode of operation; 
FIG. 43C illustrates a flow chart for the relay operation handoff for FIGS. 43 A 
and43B; 

30 FIG. 44 shows.an example of a display in accordance with the present invention, 

wherein a road map is shown indicating the location of the user's vehicle 4400, locations 
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of junctions 4410 superimposed on the road map, and known emergency vehicle 
locations 4430; and 

FIG. 45 illustrates a display provided responsive to information from any of the 
plurality of received signals, and illustrates a directional indication 4510 superimposed on 
5 the display indicating to the user that the most efficient way to bypass this particular 
congestion would be, in the illustrated example, to drive to the left 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT: 

While this invention is susceptible of embodunent in many different forms, there 

10 is shown in the drawing, and will be described herein in detail, specific embodiments 
thereof with the understanding that the present disclosure is to be considered as an 
exemplification of the principles of the invention and is not intended to limit the 
invention to the specific embodiments illustrated. 

FIG. 1 is a system diagram of one embodiment of the present invention. Vehicle 

15 100 comprises sensing means 110 afHxed to support means 120 affixed to vehicle 100, 
wherein the sensing means couples data to processing means 130 and display means 140. 
Data sensed by sensing means 110 and processed by the processing means 130 is relayed 
via logical display link 150 to display means 144 displayed to the user. The processing 
means can alternatively be positioned with the display to receive and process the sensed 

20 data for display. 

The display means may comprise a CRT display such as manufactured by major 
consumer electronics manufacturers including Sony Corp. (Japan), JVC Corp (Japan), 
Zenith Corp. (Chicago, IL), and ASC Systems (St. Claire Shores, MI). Other display 
technologies such as liquid crystal displays available through vendors such as NEC 

25 Electronics (Santa Clara, CA), Toshiba America Electronic Components (Irvine, CA), 
and King Bright Corp. (City of Industry, CA); Plasma displays are available from ASC 
Systems, and Displays Inc. (Lev^ston, PA); and LED displays such as provided by 
Colorado Microdisplay Inc. (Boulder, CO), King Bright Corp., and Bivar Inc. (Irvine, 
CA.). 

30 FIG. 2 illustrates a structural diagram of an image sensor in accordance with the 

present invention. Camera body 200 comprises image sensor means 210, which may be a 
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CCD array, or other image sensing device. Camera body 200 also comprises lens 220. 
Sensing of a physical object 230 is performed by lens 220 gathering reflected light from 
the object 230 and projecting the image 240 of that object on the image sensing means 
210. 

5 In preferred embodiment of the invention, the image sensing means further 

comprises a moveable filter as shown in FIG. 3. FIG. 3 is a structural drawing of an 
image sensor wherein camera body 300 comprises image sensor 310 and lens 320 and an 
infrared filter 325. In operation, the infrared filter 325 may be in a bypass position 330 
or in an active position 340, rotating pivotally around pivot 350. In this preferred 
10 embodiment, the CCD array which is sensitive to both visible light spectrum and infrared 
spectrum may optionally have an infrared filter interposed between said sensor and lens 
assembly 320, thereby restricting or eliminating the infrared spectrum from arriving at the 
image sensor thereby limiting the sensor to only sensing visible light. Alternatively, the 
filter may be deployed in the bypass position thereby allowing infrared energy to travel 

1 5 through lens 320 and impinge incident upon sensor 310. CCD arrays suitable for this 
invention are available from a number of commercial sources, including Edmund Scientific 
(Harrington, NJ), Hewlett Packard (San Jose, CA), Samuel Video-Components USA 
(Rochelle Park, NJ), and Orbit Semi-conductor (Sunny Valley, CA). 

FIG. 4 shows another embodiment of the invention indicating the use of a vehicle 

20 speed sensor input to control a retractable support. Vehicle 400 comprises a sensing 
means 410, a retractable support means 420, a processing means 430, a display means 
440, and a wiring harness 460. Image data is processed for display by the processor 430 
and is coupled (wirelessly or via wiring harness) to the display 440. Alternatively, data 
can be conveyed via logical display link 450 from sensing means 410 to display 440. As 

25 shown in FIG. 4, wheel speed sensor 470, or engine computer 480, may be coupled via 
wiring harness 460 to the retractable support means 420. This permits either or both of 
wheel sensor 470 and engine computer 480 to operatively control retractable support 
means 420 responsive to vehicle speed. 

FIG. 5 shows a detailed view of an alternate embodiment of a pan and tilt 

30 mechanism in accordance with the present invention. Camera body 500 is mounted on an 
assembly comprising a tilt mechanism and a pan mechanism. The tilt mechanism 
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comprises a tilt motor 520 driving a tilt pinion 530 which operates in cooperation with a 
tilt rack 540. The pan motor 540 operates a pan pinion 560 which operates in 
cooperation vAih a pan rack 570. The support means comprising the pan and tilt 
mechanism is attached to support 580. Operating the tilt motor 520 in a forward;^:., 

5 direction will tilt the camera body 500 through a tilt axis thereby positioning lens 

assembly 510 of the camera body 500 to view or sense image data above the center line 
of site of the camera. Operating the tilt motor 520 in a backward direction will tilt the 
camera body 500 through a tilt axis thereby positioning lens assembly 510 of the camera 
body 500 to view or sense image data below the center line of site of the camera Pan 

10 motor 550 permits the image sensor assembly 500 to rotate around a vertical axis 

coincident with the axis of support 580 thereby allowing the camera body 500 and lens 
assembly 510 to face in various heading directions with respect to the support 580. 

Motors and actuators suitable for use with the pan and tilt mechd|||^^^^e 
present invention are available from a number of manufacturers including vendors su^^u5I£^^-,n^ 

15 MGCInc. 

FIG. 6 is a schematic diagram of the one embodiment of the present invention in 
operation. A road hazard 600 is present on a thoroughfare ahead of a first vehicle 610, a 

0 

second vehicle 650. and an intermediate vehicle 660. The first vehicle 610 compnses a 
first sensing means 620 and a first transceiver means 630. The first sensing means 620 is 

20 coupled to a first display means 623 via first logical display link 62g|^^ 

corresponding to information sensed by the first sensing means 620 is transmitted by the 
first transceiver means 630 as a first transmitted signal 635. The second vehicle 650 
comprises a second sensing means 640 and a second transceiver means 670 both of which 
are operatively coupled via a second logical display link 690 to a second display means 

25 680 within second vehicle 650. The first transmitted signal 635 carries information 
content that is displayed on second display means 680 responsive to the reception by 
the second transceiver means 670. This permits a display generation on the second 
display means 680 irrespective of the fact that intermediate vehicle 660 may be blocking 
a direct line of sight between the second vehicle 650 and the rj^^^ard 600. 

30 Radio frequency transceivers suitable for use with the preseh^^vention are 

avsulable from a variety of vendors including Motorola Corp. (Schaumburg, IL). 
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FIG. 7 is a schematic drawing of another embodiment of the present invention in 
operation where road hazard 700 exists on a thoroughfare ^ead of a first vehicle 745, a 
second vehicle 765 and a third vehicle 775. A stationary image sensing means 720 
, additionally comprising a stationary transceiver means 730 are fixed to stationary 
S support 710, which in the illustrative embodiment, can be a road sign. The stationary 
transceiver means 730 transmits a transmitted signal 735 in the direction reversed to that 
of traffic flow, and/or to a logical central controller which displays and/or rebroadcasts 
the signal. The first vehicle 745 comprises a first vehicle transceiver 750 and a first image 
sensing means 740: The first vehicle 745 also comprises a first vehicle display 748 

10 coupled to the first vehicle transceiver 750 and the first image sensing means 740. The 
first vehicle's transceiver 750 produces a first vehicle's transmission 755 which is 
transmitted opposite of traffic flow. Second vehicle 765 comprises a second vehicle 
receptiojL and sensing means 760 logically linked to second vehicle's display 768. The 
' second vehicle's reception and sensing means 760 has the ability to receive the first 

IS vehicle's transmission 755 and the stationary transmitted signals 735 and to discriminate 
between the two received signals and to generate a display on the second vehicle's 
display 768 responsive to the data in the transmitted signal 735 and the data in the first 
vehicle's transmission 755. In an analogous manner, the third vehicle 775 has a third 
vehicle's reception and sensing means 770 coupled to the third vehicle's display 778 and 

20 is also capable of independently receiving and discriminating transmitted signals 735 and 
first vehicle's transmission 755 for generating a display responsive thereto. The 
selection and discrimination means is detailed later in FIG. 32. 

FIG. 8 is an illustration of a display in accordance with the present invention, 
illustrating multiple images from ahead in the traffic flow and operator controls. The 

25 display means 800 includes a display comprised of a textual information window 810, a 
first visual image 820, and a second visual image 830. The first visual image 820 may 
have a first legend 822 superimposed upon said image. In an analogous fashion, the 
second visual image 830 may have a second legend 832 superimposed upon it. Legends 
822 and 832 may be produced by a character generator circuit such as available by 

30 Hypercieption Inc. (Dallas, TX). Alternatively, general purpose display driver integrated 
circuits are available from companies including Telcom Semiconductor, Inc. (Mountain 



NSOOCID: <WO. 



.0143104A1J_> 



wo 01/43104 PCTAJSOO/33433 

- 12- 

View, CA), Silicon Motion, Inc. (San Jose. CA), Toshiba America Electronic 
Components (Irvine, CA), Harris Semiconductor (Melbourne, FL), Chip Supply, Inc. 
(Orlando, FL), Maxim Integrated Products (Sunnyvale, CA). and NEC Electronics (Santa 
Clara. CA). The legends 822 and 832 may comprise identification indicating th*fel5£)urce 
5 of the visual image and/or additionally may comprise other information such as 

transmitted by the source of the information or generated internally by the processing 
means of the present invention. The user controls of display means 800 include a tilt 
control 840, a pan control 850, a zoom control 860, and a focus control 870. In the 
illustrated embodiment, these controls are shown as thumbwheels permitting the user to 
10 easily adjust respectively tilt, pan, zoom, and focus of the image sensing means. In an 
alternate embodiment these controls may be implemented as buttons, or as graphical user 
interface elements on an interactive touch screen display. 

Touch screens are commercially available from a number of manuf^c^^r^j. 
including Microtouch Systems, Inc (Methuen, MA), Omron Electronics (Schaumburg, 
15 IL), MSI Technologies LLC (Englewood, CO), Advanced Input Devices (Couer D' Alene, 
ID). (Eden Prairie, MN), Micro Mo Electronics (Clearwater, FA), Nidec America 
(Canton, MA), and Dado Corp. (Summerset, NJ). 

FIG. 9 is a detailed schematic illustration of a retractable support as one 
embodiment of the present invention. Vehicle 900 comprises a retractable support 
20 means 940 which provides for positioning an image sensing means 910 in eithei^of a . . 
retracted position 920 and a deployed position 930. 

FIG. 10 is an illustration showing additional detail of the mechanism of FIG. 9 
wherein the retractable support comprises a retractable support housing 1040 that emits 
a retractable support column 1040, upon which is mounted a pan and tilt means 1020, 
25 which is in turn affixed to an image sensing means 1000 having an angular field of view 
1010. 

FIG. 1 1 is an even more detailed view of the image sensing means and tiie pan and 
tilt means of FIG. 10. As shown in FIG. 1 1, the image sensing means 1100 has a first 
line of sight 1 1 10 and a second line of sight 1120 comprising a tilt range 1 130 between 
30 the first and second line of sights, respectively. In a similar fashion, tfee'fiifage-sensing - 
means has a pan range 1140 as illustrated in FIG. 11. The image sensing means 1100 is 
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thereby provided means to tilt the line of sight within the tilt range 1130 and to pan in 
different directions within the pan range 1140 relative to an axis extending through 
support column 1 ISO. The tilt mechanism permits the image sensing means to look up 
and look down and the pan means permits the image sensing means to rotate and 
5 essentially "look around." 

FIG. 12 is a schematic illustration showing one embodiment of the present 
invention utilizing a speed sensor input similar to the one shown in FIG. 4, As shown, 
vehicle 1200 comprises a sensing means 1210 affixed on a retractable support means 
1220 to the vehicle 1200. Data from the sensing mean 1210 is relayed via the sense data 

10 relay link 1225 to the processing means 1230. The processing means then produces 

image data responsive to the data from the sensing means and relays the image data via an 
image data relay link 1235 to display means 1240. The vehicle's speed sensing means 
1250, which in the illustrated embodiment, is a vehicle wheel speed sensor, is coupled to 
a retractable support controller 1260. The retractable support controller 1260 issues a 

15 control signal via control signal link 1270 to the retractable support means 1220 which 
responds operatively thereto. In the illustrated embodiment, the operation of the 
retractable support means is thereby responsive to the speed of the vehicle as detected 
by the vehicle speed sensing means 1250. In practice, this can be used to either deploy, 
or retract, the camera if the vehicle speed approaches certain thresholds which are 

20 programmably determined. For example, the system may prevent the image sensing 
means from being deployed on the retractable support by instructing the retractable 
support to retract if the vehicle's speed is above 5 mph. In an alternative embodiment, 
the retractable support may deploy to a preset point if the vehicle's ground speed is 
under 30 mph then deploy even further if the vehicle's speed is above 30 mph. 

25 FIG. 13 shows an alternate embodiment of the speed sensing retractable support 

controller means similar to those shown in FIGS. 12 and 4. The image sensing means 
1310 is affixed to a retractable support means 1320 to the vehicle 1300. The vehicle 
1300 also comprises processing means 1330 and display means 1340. The image sensing 
means 1310 additionally comprises a windspeed detection apparatus 1350 coupled to a 

30 windspeed detection controller 1360. The windspeed detection controller 1360 issues 
control signals to the retractable support 1320 via link 1370. The link 1370 permits the 
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operation of the retractable support 1320 to be controlled responsive to the detected 
ambient wind condition. The link 1370 can be direct control (wired or wireless) of the 
retractable support, or can be coupled to processing means 1330 which controls the 
retractable support. In the illustrative embodiment, this can give a more ac^i^te 

5 indication of the tnie wind force that might be acting upon the image sensing means 1310 
and the retractable support 1320 because it includes information that would not be 
available from an engine computer or ground speed sensor as shown in FIGS. 4 and 12. 

FIG. 14 shows an alternate embodiment of the retractable support wherein the 
retractable support 1420 variably positions the image sensing means 1410 illustrating the 

10 option of off to the side of vehicle 1400. 

FIG. 1 5 shows one embodiment of the present invention wherein the retractable 
support structure is activated by means of hydraulic piston.: ^"^^^ sensing means 1510 
is affixed to one end of a boom 1520 which is affixed at its opposife'^^^il^^to vehicle 
1500. Hydraulic cylinder 1524 positioned between vehicle 1500 and boom 1520 j>ermits 

15 the boom 1520 to be raised and lowered around pivot 1522 with respect to the vehicle, 
Tilt means 1526 compensates for the motion of the raising and lowering of the boom by 
rotating image sensing means 1510 correspondingly. Image sensing means 1510 provides 
data to processing means 1530 which then provides image data to display means 1540 
for display to the user. 

20 FIG. 16 shows a schematic drawing of a multi-^^d^j^l^^^^n one 

embodiment of the present invention ^^erein there are a plurality of linage sensing means 
1610, 1615, and 1620, and other sensing means 1625 coupled to processor 1630. 
Processor 1630 responsive to signals from the various sensing means produces a display 
output signal for display to the user on display means 1640. As shown in the 

25 illustration, some of the image sensing means (such as image sensing means 1610) may be 
supported on the vehicle 1600 via retractable support 1650. Alternatively, some of the 
image sensing means (for example, 1615 and 1620, as illustrated) are supported via a 
fixed support means (respectively, 1660 and 1670). In the illustrated embodiment, 
sensing means 1625 is an antenna connected td^^^l^etying meayg|s. 1680. The antenna 

30 may be fixed or retractable. The processing means 1630 includes means to discriminate 
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the various signals supplied to it from various sensing means and to select portions of 
those signals for integration into a display integration for the user. 

FIG. 17 shows an embodiment of the present invention as utilized on watercraft. 
Watercraft 1700 is shown with image sensing means 1710 mounted on support mast 
5 1720. The image sensing means 1710 is coupled to processing means 1730 and thence to 
display means 1740 for producing an integrated display presentation for the user. 

FIG. 18 shows details of one implementation of the mounting means as in FIG. 
17. Image sensing means 1810 mounted on support mast 1820 is capable of the pan and 
tilt motions as shown previously in FIGS. 5 and 11, The tilt mechanism permits locating 

10 the image sensing means line of sight 1870 variably up and down through range 1850 as 
shown in the illustration. The pan mechanism, as previously disclosed, permits image 
sensing means line of sight to be rotated through a range 1830 as shown on the 
illustration iR-the direction 1840 as shown by the arrows: However, given the nature of 
water craft, an additional axis of freedom is supplied, that freedom namely being the 
. 15 ability to roll along the axis of the line of sight 1870 on the directions 1860 as shown by 
the arrows. By permitting the image sensing means 1810 to be additionally adjusted in 
the direction 1860 of rotation, it permits the image sensing means 1810 to compensate 
for the variable roll of the watercraft. 

FIG. 19 shows one embodiment of the present invention wherein the system 

20 comprises a receiver 1970 coupled to a processing means 1930 which is in turn coupled 
to a display means 1940 within the vehicle 1900. Also ccnipled to the receiving means 
1970 is antenna 1960. The antenna 1960 provides for receiving transmissions from 
sources external to the vehicle, decoding of the received transmissions via the receiver 
1970, and passing said decoded data to processing means 1930 for integration into a 

25 display presentation for the user. 

FIG. 20 shows an alternate embodiment of the present invention wherein the 
system comprises an antenna 2060 coupled to receiving means 2070, which in turn is 
coupled to processing means 2030 which is, in turn, coupled to a display means 2040. In 
the illustrated embodiment, signals are received by antenna 2060 from sources outside the 

30 present system and are decoded by receiving means 2070 producing data that is then 
relayed to processing means 2030 for integration into a display presentation on display 



«IS0OaD: <WO 9143104A1 J_> 



wo 01/43104 PCT/USOO/33433 

-16- 

2040 to the user. As shown in the drawing, the system is not required to be physically 
present within a vehicle. 

FIG. 21 shows another embodiment of the present invention as used in a central 
station. A plurality of signal receiving means (such as 2160, 21fi5, and 2168) are coupled 
S to a receiving decoder 2170. Examples include a radio frequency antenna 2160, a 
microwave antenna 2165, and telecommunications links 2168. The receiving and 
decoding means 2170 relays received and decoded data to the processing means 2130. 
The processing means 2130 generates data for a plurality of individual display elements 
2145 that comprise the display means 2140. This permits an operator to receive 
10 information communicated from a variety of sources (as shown in the illustrated example, 
telecommunications, radio frequency, and microwave links) and to have the system 
integrate that data and present it for display. The system of the present invention is npt 
limited to the communications link varieties as shown. Other additipnal communication 
links include infrared communications, satellite communications, fiber optics, etc. 
15 Examples of manufacturers providing components for microwave 

communications include Sprague-Goodman Electronics (Westbury, NY), Atlantic Coast 
Instruments (Brick, NJ), and Microwave Communication Laboratories (St. Petersburg, 
FL). The individual display elements 2145 in the illustrated embodiment may comprise 
any combination of CRT displays, LED displays, LCD displays, plasma displays, or 
20 other types of displays. 

FIG. 22 shows an alternate embodiment of the present invention wherein the 
image sensing means 2210 is affixed to vehicle 2200 via the fixed support 2220. The 
image sensing means 2210 relays signals to processor 2230 ^^ch decodes signals and 
produces an integrated display presentation for display to the user on display 2240. In 
25 the illustrative embodiment, the image sensing means 2210 has a fixed field of view 2250 
determined by the mounting of image sensing 2210 on support 2220 and the relative 
orientation of vehicle 2200. 

FIG. 23 shows another embodiment of the present invention wherein the image 
sensing means 2310 may be operatively elevated above the position of vehicle 2300 >da 
30 the support element 2320, thereby permitting the field of view 2350 to encompass 
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objects such as vehicle 2360 which would otherwise be invisible to the operator of 
vehicle 2300 due to interfering vehicle 2340. 

FIG. 24 shows a schematic of one multiple vehicle embodiment of the present 
invention. A first vehicle 2400 comprises a image sensing means 2410, a processing 
S means 2430, a display means 2440, and transmission means 243S. ,The image sensing 
means 2410 having a field of view 2450 senses data corresponding to an image and 
supplies that information to transmitter 2435. Transmitter 2435 transmits said 
information via transmitted signal 2437 to the antenna 2460 of second vehicle 2490. The 
received signal via antenna 2460 is relayed through receiving means 2470 which receives 
10 and decodes the data in transmitted signal 2437 and provides that data to processing unit 
2475. Processing unit 2475 generates a local display presentation for the user which is 
displayed on display means 2480. In the illustrated embodiment, the image sensing 
means 2410 and transmission means 2435 are separate means that are coupled. 

FIG. 25 illustrates an alternate embodiment of the first vehicle as shown in FIG. 
15 24. The combination image sensing means and transmission means 2590 is affixed to 
vehicle 2500 via support 2520. The combination image sensing and transmission means 
2590 provides a transmitted signal 2595 as well as a signal coupled to processing element 
2530. Processing element 2530 produces an integrated display presentation on display 
means 2540 for the user. As shown in the illustrated embodiment, the signal 
20 transmission means and the image sensing means may be combined into one unit. 

FIG. 26 shows an alternate embodiment of the present invention wherein 
emergency vehicle 2600 is equipped with a image sensing and transmission apparatus 
2690 which provides a transmitted signal 2695. The transmitted 2695 is received by 
antenna 2660 and decoded by receiver 2670 for processing by processing element 2680 
25 and subsequent display to the user of a integrated display presentation on display means 
2685, said display means located in or on second vehicle 2650. This embodiment 
illustrates the usefulness of the present invention wherein the emergency vehicle 2600 
can relay important information regarding traffic or road conditions that are significantiy 
ahead of the user in the vehicle 2650. 
30 FIG. 27 shows an alternate embodiment of the present invention wherein a 

number of image sensing means 2710, 2720, 2730 are arranged at strategic location along a 
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roadway 2700. The data relayed from image sensing means 2710, 2720, and 2730 are 
processed by central station 2740 and the information content is then is relayed via 
transmitter over transmitted signal 2760 to one or more vehicles 2770 equipped to 
receive said signals. This permits the central station to consolidate information from a 
5 variety of sources and to present the consolidated information, or alternatively, the raw 
information from the image sensing stations, to vehicles on the roadway. Additionally, 
or alternatively, the individual sensing means can further comprise transmitter 
subsystems and provide for direct communication to the vehicles. As shown in FIG. 21, 
the central station may comprise a plurality of different types of receiving means, and a 
10 plurality of individual displays, permitting an operator to observe and control the 

operation of central station 2740. Also, as shown in FIG. 27, the image sensing sources 
2710, 2720, 2730 may deployed on a wide variety of supports. In the illustrated 
embodiment, image sensing means 2710 is deployed on top of a building structure 2715 

c 

located adjacent to the roadway 2700. Also, as shown in the illustrated example, image 

15 sensing means 2720 is located on a sign overpass 2725 which bridges roadway 2700. 
Alternatively, as shown in the illustrative embodiment, image sensing means 2730 may 
simply be located adjacent to roadway 2700. 

FIG. 28 illustrates schematically the central station 2805 comprising a variety of 
sources of information including textual information 2800 and visual information 2810. 

20 The visual information may be derived from roadside image sensing means, such as those 
illustrated in FIG. 27, or mobile image sensing means, such as those illustrated in FIG. 
26. Additionally, the textual information may be generated internally to the central 
station 2805, or externally from the central station. The sources of information are 
assembled by processing unit 2820 into a signal to be transmitted by transmitter 2830. 

25 The transmitted signal, 2835 conveys any combination of visual and textual information 
from the central station 2805 to the user's vehicle 2840. The user's vehicle 2840 
comprises an antenna 2850 coupled to a receiving and decoding means 2860 and a 
processing means 2870. The processing means 2870 is the further coupled to a display 
means 2880. The processing means 2870 has the ability to interpret the various types of 

30 information conveyed by transmitted signal 2835 and to selectively incorporate elements 
of that transmitted signal into an integrated display presentation 2890. The integrated 
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display presentation 2890 can comprise one or both of visual information 2893 
corresponding to a processed version of the visual content 2810 and an overlay of 
processed textual information 2896 corresponding to the textual information 2800. The 
system of the present invention thereby permits the user of the system to discriminate 

5 and interpret a wide variety of data transmitted to the user's vehicle. 

FIG. 29 illustrates another embodiment of the present invention, wherein an 
image sensing means 2910 is afHxed via support structure 2920 to a road sign 2930 
located adjacent the right-of-way 2940. The information content sensed by image 
sensing means 2910 is then transmitted via transmitter 2960 to vehicles such as the 

10 user's vehicle 2980, via the transmitted signal 2970. The transmitted signal 2970 is 

received, decoded, and processed within the user's vehicle 2980 and displayed on display 
2990. In the illustrated embodiment, the information content relayed from image sensing 
means 2910 and thus displayed on the user's display means 2990 permits the user to 
make a decision on whether to stay-on-right of way 2940 or to take exit 2950 responsive 

15 to the traffic conditions that are thus visually observed on display means 2990. 

FIGS. 30 A and SOB are representative examples of the types of display 
information that may be presented by an embodiment of the present invention. FIG. 
30A illustrates the display of a schematic overview of 3010 of traffic congestion in the 
vicinity of road hazard 3020. In addition to the schematic representation of 3010, other 

20 areas of information that may be included in the display comprise a warning, or 

advisement area 3045, and a user control area 3055. The warning or advisement area 
3045 may comprise urgent wamings displayed in a textual or graphic form 3030 and 
helpful information displayed independently in a textual or graphic form 3040. 
Additionally, the user control area 3055 may comprise a plurality of user controls 3050, 

25 which control the operation of the system and/or modify attributes of the display 

presentation. In the illustrated example, these user controls include the ability to display 
distances in miles or kilometers. 

FIG. BOB shows an altemate display presentation of substantially the same 
information as shown in FIG. 30A wherein the display comprises a visual representation 

30 3070 of the area in which the traffic congestion occurs responsive to road hazard 3020. 
In the illustrated embodiment, superimposed on top of the visual presentation 3070 are 
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usefiil warnings in a textual or graphical form including speed warning 3030 and 
directional information 3040, Additionally, user control elements such as user control 
3050 may be superimposed on top of display presentation. 

FIG. 3 1 A is a schematic representation of an alternate embodiment of the present 

5 invention wherein a combination image sensing and transmission apparatus 3110 is 

located adjacent the roadway 3170. The apparatus 3110 transmits data representative of 
the image sensed by the image sensing means via transmitted signal 3120 to receiver 
3130. Receiver 3130 relays the transmitted and decoded data to processing means 3140 
which produces an integrated display presentation for display on video billboard 3150. 

10 Video billboard 3150 then provides a display usable by users in one or more vehicles 
3160 on the road 3170 thereby giving the operators of said vehicles advance warnings of 
traffic conditions as observed by image sensing means 3110. Note that in the illustrative 
embodiment, no portion of the system need be required to be located wthin or on the 
user's vehicle. In addition, the transmittal signal 3120 can be received by a receiver 

15 within a vehicle and processed and displayed within the vehicle (such as shown in FIGS. 
1. 6, 7, 8, etc.) 

FIG. 3 IB illustrates another embodiment of the invention similar to that depicted 
in FIG. 3 1 A wherein the image sensing means 3115 is directiy coupled to a processing 
means 3145 which is in turn directiy coupled to billboard 3150 thereby providing for 
20 display to integrated information display through the users responsive to the image 

detected by image sensing means 3115 of the roadway 3180. Again, as in FIG. 3 1 A, note 
that no portion of the present invention needs to be physically present in any of the 
users' vehicles. 

FIG. 32 shows one embodiment of the signal receiving and processing element in 
25 accordance with the present invention. As shown in FIG. 32, one or more antennas 3200 
and 3201 are coupled via connections 3205 and 3206 to receiver/decoder 3210. 
Receiver/decoder 3210 then decodes the signals thus received and relays them to 
processing means 3350 via a plurality of data connections 3220. Optionally, one or more 
image sensing means 3223 may coupled via data connections 3226 to the processing 
30 means 3350. The processing means 3350 is further comprised of a selector subsystem 
3360 and formatting and processing subsystem 3340. The selector subsystem 3330 
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selects firom the available data inputs 3220 and 3226 and produces one or more selected 
output signals 3333 and optionally 3336 for input by the processing and formatting 
means 3340. The processing and formatting means accepts the plurality of selected data 
signals and performs processing and formatting operations on that data producing an 
5 integrated display presentation. The integrated display presentation is conveyed to 

display 3360 (for presentation) via link 3345. Processing and formatting operations may 
include operations such as superimposing textual information on top of video 
informattbn, producing schematic displays of traffic patterns and congestion, producing 
warning information responsive to signals received via antennas 3200 and 3201, and 

10 integrating received information with data signals from image sensing means 3223. In a 
preferred environment, the receiver 3210 includes within each of the output signals 3220, 
an identification of which of the antennas (3200, 3201) the resultant data signal was 
received from. This in turn permits the processing means 3350 to discriminate between 
the different data signals received on the basis of transmission methodology and 

15 identification. 

FIG. 33 represents a simplified block diagram of the receiving and processing 
systems of the present invention. Shown in FIG. 33, the antenna 3300 receives signals 
broadcast to it and conveys those signals via link 3305 to receiver 3310. Receiver 3310 
then relays the received signal through decoder 3320. The receiver 3310 and decoder 

20 3320 together comprise receiver/decoder 3370. The output of the receiver decoder 3370 
is a data signal that is then coupled to the image processing and format means 3350. The 
image processing and format means 3350 then integrates the data signals as received into 
a display and then couples that display information to display means 3360 for display to 
a user. Specific commercially avmlable receivers, decoders, image processing and 

25 formatting subsystems and displays as detailed above herein are also applicable to FIGS. 
32 and 33. 

FIG. 34 shows one embodiment of the present invention showing detail of the 
image processing and formatting means 3350 as shown in FIGS. 32 and 33. In this 
embodiment, the received data is representative of an impaired visual that has noise 
30 patterns established within it as shown in visual 3410. This impaired visual is 

characteristic of signals received via conventional wireless technology in the environment 
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of Other transmitters or other sources of electromagnetic interference. The impaired 
visual 3410 is then processed by the image processing and format means 3350 and 
specifically undergoes a noise reduction step 3450 producing a restored image 3420 
which is representative of the image content without the noise imposed by interference. 
5 FIG. 34 shows one example of types of image processing that can be performed by the 
image processing and format means 3350. Other examples as cited elsewhere in the 
specification include enhancing the contrast, adjusting brightness, adjusting color balance, 
filtering to ren)ove impulse noise, removing artifacts that may be brought upon by the 
particular kind of image sensor that is being used, spectrum conversion from a non-visible 
10 to a visible spectrum, etc. 

FIG. 35 is a schematic representation of the multiple transmitter embodiment of 
the present invention wherein the user's vehicle 3500 receives a plurality of signals via 
antenna 3510 from sources including a first transmitter 3520 and a second transmitter 
3530 wherein each of the transmitters 3520 and 3530 comprise image sensing means 
15 directly coupled to signal transmission means. The transmitter 3520 transmits data 
representative of the image perceived in field of view 3560. The transmitter 3530 
transmits data representative of the field of view 3550. The user's vehicle 3500 on right 
of way 3540 receives signals via antenna 3510 and the processing and formatting means 
3512 collects and integrates the received signals into an integrated display presentation 
20 which is then displayed to the user on display means 3514. The display may include a 
representation of a picture in a picture wherein the field of view 3560 is cont^ned within 
the image of the field of view at 3550 or vice versa, or may comprise altematingly 
switching between the visual received from transmitter 3520 and the visual received from 
transmitter 3530, or in yet another alternate embodiment, may include a split screen 
25 mode of operation where visuals received from each of the transmitters 3520 and 3530 
are shown on respective portions of the display screen 3514. 

FIG. 36 shows an schematic illustration of the operation of an embodiment of the 
present invention wherein a building 3630 blocks the view of roadway 3640 some 
distance from the user's vehicle 3600. The present invention includes combination 
30 sensing and transmitting means 3620 on or near building 3630 having a field of view 3660 
which includes that portion of roadway 3640 that is obscured from the user. The 



wo 0 1/43104 PCT/USOO/33433 

-23- 

combination transmitter sensing means 3620 transmits a signal 3625 that is received at 
antenna 3610 of the user's vehicle 3600. The received signal is decoded and processed 
by processing means 3612 and the processing means 3612 produces an integrated display 
for presentation to the user on display means 3614 within the vehicle, wherein the 
S presentation is of the roadway 3640 in the area covered by the field of view 3660. 

FIG. 37 shows a schematic representation of the one embodiment of the present 
invention as used in watercraft. Watercraft 3700 incorporates an antenna member 3710 
which is coupled to a receiver 3720. Receiver 3720 is in turn coupled to a processing and 
integration me^ns 3730 which responsive to the data from receiver 3720 produces an 

10 integrated display for presentation for the user. The integrated display data is 
communicated to display 3740 for presentation thereupon. In the illustrated 
embodiment, the system of the present invention permits the operator of the watercraft 
to have the benefit of remotely located image sensing means, for example, giving a visual 
of an approach to a harbor or a docking area, or navigation of a difficult stretch of 

IS waterway. The image sensors may also or alternatively include infrared or other types of 
information derived from radar, such as maritime radar, thus permitting the generation of 
a schematic view of a watercraft within a waterway to be displayed on the display means 
3470 without requiring the user's watercraft 3700 to possess radar means. 

FIGS. 3 8 A, 38B, 38C represent one embodiment of the present invention. FIG. 

20 38A is a schematic representation of traffic congestion involving an emergency vehicle 
3800. The user's view (from vehicle 3810) is blocked by intermediate vehicle 3820 
which is located between the emergency vehicle 3800 and the user's vehicle 3810. 
Utilizing the system of the present invention, image means 3830 affixed to support 
means 3840 produces a display on display means 3850 on the interior of the user's 

25 vehicle 3810 thereby providing a display on the display means of areas that would 

otherwise be obscured from direct line of sight of the user in vehicle 3810 by intermediate 
vehicle 3820. 

FIG. 38B shows a representation of one possible display as shown by display 
means 3850 wherein the information received by image sensing means 3830 is directly 
30 displayed for the user. 
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FIG. 38C shows a different display that would be presented to the user on 
display means 3850 which comprises a schematic view derived from information received 
from image sensing means 3830 and processed to produce the overhead schematic view 
and other user information as shown in the illustration. 
5 FIGS. 39A and 39B represent an alternate embodiment of the system as depicted 

in FIGS. 38A, 38B, and 38C wherein the emergency vehicle 3800 is again blocked from 
the direct view of the user in vehicle 3810 by intermediate vehicle 3820. As in FIGS. 
38A, 38B, and 38C. the image sensing means 3830 is able to provide the user a view 
similar to that shown in FIG. 38B of the traffic congestion ahead of the user's vehicle. 
10 Further, tiie user' s vehicle is equipped witii a receiving antenna 3870 coupled to receiver 
and processing means 3860. Receiving and processing means 3860 combines tiie image 
information received from sensing means 3830 with tiie transmitted signal 3885 from 
combination sensing means and transmitter 3880. located adjacent to the roadway as 
shown in the illustration. The combined information may be presented to the user in the 
15 form as shown in the FIG. 39B. wherein a smaller inset portion of tiie display includes a 
visual representation of data from sensing means 3830. as shown in 3900. and also shows 
a schematic view of the traffic congestion as image 3910. In addition, textual and graphic 
information 3920 can be superimposed on the combined display. 

FIGS. 38A-C and FIGS. 39A-B therefore demonstrate, inter alia, in accordance 
20 with the present invention, one or more sources of information content can be utilized in 
generating displays. The types of displays that can be generated include a plurality of 
direct visual representations, generated schematic representations, textual information, 
graphical information, and any combination of the above. 

FIG. 40 is another exemplary embodiment of the present invention, wherein a 
25 first vehicle 4000 comprises a support means 4010, image sensing means 4020, and a 

transmission means 4030. The image sensing means 4020 relays data to transmitter 4030 
which in turns transmits said data as te^ansmitted signal 4035. Transmitted signal 4035 is 
received by a plurality of vehicles 4065 and 4075 via antenna means 4060 and 4070, 
respectively. In vehicle 4065, the signal received via antenna 4060 is processed by 
30 processing means 4068 and displayed for a user on display 4062. In vehicle 4075, the 
signal received by antenna means 4070 is processed by processing means 4078 and 



BNSOOCIO <WO OimiddAi I ■» 



Wo'oi/43104 PCT/USOO/33433 

.25- 

displayed for the user on display means 4072. As illustrated in FIG. 40, the present 
invention permits the operators of vehicles 4065 and 4075 to have the benefit of the field 
of view of image sensor 4020 notwithstanding the fact that intermediate vehicle 4045 
blocks their direct line of sight 
5 FIG, 41 shows one embodiment of the present invention which utilizes multiple 

display means, 4110, 4120, and 4130, all coupled to processing means 4140. The 
processing means 4140 is also coupled to receiving means 4150 and image sensing means 
4160. The processing means 4140 operates on data supplied by image sensing means 
4160 and data supplied by receiving means 4150 to generate a plurality of integrated 
10 displays for the user. Each of these integrated displays is routed to one or more of the 
display means 4110, 4120, and 4130. This embodiment of the present invention permits 
a user to see multiple views of a traffic congestion situation and to receive more 
information simultaneously than would be able to be conveyed via a single display 
means. In one embodiment, display 4110 may display a >asible light image of the data 
15 received from image sensing means 4160 and display means 4120 may display a visual 
light image representation of an infrared scene sensed by image sensing means 4160. 
Note that although a single image sensing means 4160 is shown on the illustration, the 
present invention also can operate when there are plurality of image sensing means, each 
providing individual respective input to processing means 4140. 
20 FIG. 42 shows one embodiment of the present invention that supports a relay 

mode of operation. In FIG. 42, image sensing means 4210 having a field of view 4200 is 
affixed to support 4215. In the illustrated example, support 4215 is located on top of 
roadway exit sign 4217, although this location of sensing means 4210 is not a requirement 
of the invention. Image sensing means 4210 relays data signals to transmitter 4220. 
25 Transmitter 4220 transmits a first signal 4225, In tiie illustrated example, signal 2245 is 
received by vehicle 4260 which is sufficientiy close to transmitter 4220 as to be in 
receivable range. The signal 4225 is received by antenna and receiving means 4230 and is 
subsequentiy repeated via signal transmitter 4240. In the illustrated example, vehicle 
4270 is sufficientiy far away from transmitter 4220 as to preclude reception of first 
30 transmitted signal 4225. However, vehicle 4270 is within reception range of the 

transmitter 4240 atop vehicle 4260. As a result, transmitted signal 4245 is received by 
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vehicle 4270 on antenna 42S0. Thus the data provided by image sensing means 4210 is 
first relayed from the first transmitter 4220 to the vehicle 4260 where it is in turn relayed 
by transmitter 4240 as a second transmitted signal before arriving at the user's vehicle 
4270. Although in the illustrated embodiment one interstitial vehicle 4260 was shown, in 
5 practice any number of vehicles may relay successive signals to other vehicles within 
their respective transmission ranges. Furthermore, where the vehicle (as in FIG. 41) has 
both it's own sensor and a receiver, and a transmitter (as in 4240 in FIG. 42), the relay 
can be additive, conveying (relaying) both received data and locally sensed data. This 
permits the invention to be used by users in vehicles significantly distant from a low 

10 power transmitter. 

FIGS. 43 A-43B illustrate land-based call operation relative to a moving vehicle. 
In FIGS. 43 A-43B, there are three land-based transmitters 4320, 4330, and 4340 each 
transmitting respective content signals 432S, 4335, and 4345, respectively. In FIG. 43 A, 
the signals 4325 and 4335 are within the range of and are received at antenna 4310 for 
1 5 display on display 4315. In FIG. 43B, the vehicle 4305 is in range of signal 4335 as 
received by antenna 4310 for display on display 4315. 

FIG. 43C is a flow chart of a control program to implement the cell handoflf 
operation for the system controller of FIGS. 43 A and 43B. 

FIG. 44 shows an example of a display in accordance with the present invention, 
20 wherein a road map is shown indicating the location of tiie user's vehicle 4400, locations 
of junctions 4410 superimposed on the road map, and known emergency vehicle 
locations 4430. Also shown are indications of travel time between junctions 4410, where 
an example outbound travel time is shown at 4440. The present invention allows the 
user to touch the location on the screen, for example the location of the emergency 
25 vehicle 4430, at which point the view corresponding to the traffic congestion in that area 
is shown as in FIG. 45. Positioning information can be locally computed or detected 
and/or based on a GPS-based system. 

FIG. 45, responsive to information from any of the plurality of received signals, 
shows a directional indication 4510 superimposed on tiie display indicating to the user 
30 that the most efficient way to bypass this particular congestion would be, in the 
illustrated example, to drive to the left. 
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FIG. 46 illustrates an alternative embodiment of the present invention, wherein a 
plurality of sources of traffic condition data communicate that data, and wherein one or 
more vehicles have receivers for receiving the communicated traffic condition data from 
one or more of a plurality of sources, and provide processing and display internal to the 
5 vehicle to provide a display of traffic conditions responsive to the traffic condition data, 
in a manner as described consistent witii the detailed description herein elsewhere. 

Referring now to FIG. 46. a plurality of sensor subsystems 4660a. 4660b. and 
4660c. respectively each having antennas 4620 provide for sensing traffic conditions, 
imaging and other information, and transmit respective traffic condition data A, B, and C. 
10 (which can also contain optional location data for the respective sensor subsystem 

originating the signal), to provide communicated traffic condition data. Additionally, an 
aircraft 4670 is illustrated as having a sensor subsystem 4672 Cimaging sensor) and 4673 
(preprocessor and/or transmitter) which provide output communication via antenna 4674 
of traffic condition data D. The traffic condition (A. B. C, D) can then be communicated 
15 to one or both of a centrally located relay or accumulation center 4650 which then 
rebroadcasts tiie traffic communication data signals A. B. C, D. and/or can be 
communicated directly from the sensor subsystems 4660a. 4660b. 4660c. and airborne 
sensors subsystem and transmitter 4672. 4673. and 4674, to directly communicate traffic 
condition data (A. B, C. D) to vehicles with receivers, such as vehicles 4610 and 4611. 
20 As illustrated, vehicles 4610 and 4611 each have respective receiving subsystems 4612 
and 4613, which provide for receipt of the traffic condition datii communications (A. B. 
C. D) for processing and display internal to the respective vehicles 4610 and 4611. The 
communicated data can be received directly from the originating data sources, and/or can 
be received from a centralized communication center 4650 or a relay network setup off" of 
25 the centralized or other type of gatiiering center 4650. in a manner consistent witii that 
described above herein relative to the relay and other gathering redistribution 
communications. Additionally, the central center 4650 can provide for local display. In 
addition to tiie traffic condition data illustrated in FIG. 46. it is to be understood tiiat any 
type of traffic conditioii data communication can be included herein, such as other types 
30 of stationary fixed sensors, other types of moving sensors, whether airt)ome or ground- 
driven, as well as buried sensors, infrared communication dati^ and radar communication 
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data. All of these sources of traffic condition data can be provided for in accordance with 
the present invention, and either the central system 4650 or the vehicles receiving the 
signal can contain the appropriate processing to provide for selection of relevant 
communicated traffic-condition data for the vehicle. These decisions can be based on user 
5 input, positional data such as GPS for the respective vehicle, relative position of sensor 
subsystems based on location data associated therewith, etc. 

Referring to FIG. 47, there is illustrated a control receiver processing and display 
subsystem, such as 4612 or 4613 in FIG. 46. Multiple sources of data communication 
are illustrated, including sensor traffic condition data 4710 (A), sensor communication 
10 data 4720 (B), traffic condition communication data 4730 (C). traffic condition data 4740 
(D). and global positioning signal 4750 (GPS) originating from satellite communication. 
Additional communicated data sources can also be provided, including radar, ground 
sensors, etc. Antenna 4760, such as on the vehicles 4610 or 4611, or the central system 
4650, all of FIG. 46, provides for initial receiving and coupling of the communicated data 
15 signals (A, B. C, D. GPS) for coupling to receiver 4770. Receiver 4770 provides for 
receiver decoding and filtering of the incoming communicated data, including source 
selection and identification, and provides decoded received data to the processor 4780. 
The processor 4780 provides for signal processing and filtering, and formatting for 
display presentation of the received decoded data. A user input device 4785. such as a 
20 keypad, joystick, touchpad. or other input means, provides for user input and partial 
selection and control to permit user control of displayed information, including selection 
of sources, types of display, etc. The processor provides display data output to the 
display 4790 which provides a display presentation in accordance with the processor 
output. The display presentation is of the type as described elsewhere herein, and any 
25 of the types of displays as mentioned elsewhere herein may be utilized, including CRT, 
LCD, electroluminescent, etc. 

FIG. 48 illustrates the software pseudo-code for the operation of the control 
system of FIG. 47. Note that ¥IG. 47 is an enhancement and alternative embodiment to 
the system as shown in FIG. 19. 
30 FIG. 48 illustrates the pseudo-code functional operation and flow for the control 

system of FIG. 47, where there is a GPS communication data signal. At step 4810, the 
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system detects the GPS system signal and feeds that signal to the receiver and processor. 
At step 4820, the processor determines the vehicle (e.g., automobile) position based on 
the GPS communicated data. At step 4830, the processor utilizes the GPS determined 
automobile position from step 4820 in conjunction with other communicated and stored 
5 data, such as other sensor signal communicated data relative locations, and computes 
which sensor subsystems are relevant for this vehicle*'for its present position, and 
determines both ahead and, where appropriate, the distance limits to receive sensor data 
from. At step 4840, the processor causes the selection of only the relevant respective 
signals from the incoming communicated traffic condition data (e.g.. A, B, C, D). 
10 At step 4850, the processor provides formatting of the appropriate data and 

processing necessary to generate a display of a particular selected relevant one of die 
sensor subsystem traffic condition data. In the illustrated embodiment of FIG. 48, this is 
shown as displaying the closest, in a forward direction, traffic conditions for display. At 
step 4860, the processor 4780 of FIG. 47, in conjunction with the user input 4785, 
1 5 provides the user with the option to select from other relevant traffic conditions for 
display. 

In a preferred embodiment of the present invention, there is a display for use by a 
vehicle operator, a means to receive signals that represent traffic conditions that the 
operator is interested in, and a processing means coupled to the receiving means that 
20 takes the information and formats in a manner suitable for display to the user. This 
system can be used, for example, to receive traffic telemetry information or estimated 
arrival times compiling traffic congestion information and displaying tiiat on a (Usplay 
that an operator in a vehicle can use it to make decisions about how to pilot that vehicle. 
The display can be any kind of display technology such as, but not limited to, liquid 

25 crystal display, (LCD), CRT, plasma display, display built of light-emitting diodes 
(LED) display, a projector driver, a half-mirrored display, or a heads up display that 
projects the image to the user directiy on the windshield or screen of the vehicle that the 
user is operating. Other methods that can be used include a half-mirror display which 
would allow the user to look through the display and see the environment directiy 

30 outside the vehicle but also see the reflection of another type of display superimposed 
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on that view. The delivery method to the user of the display can result in printing a 
representation of the traffic conditions of the traffic conditions on paper. 

Another method of display is an audible system, which, upon receipt of 
information about traffic conditions, speaks selected portions of that information to the 
5 user in a language that the user understands. <- 

All of these methods result in information, received from outside of the vehicle, 
being received by the user and allowing the user to make decisions about the operation of 
the vehicle based on that information. The direction of primary interest to the user is 
obviously directly in front of the user or in the direction of travel of the user, but it isn't 
10 always the case that only the forward direction is interesting. It may be significant for 
the operator of the vehicle to know that an emergency vehicle is somewhere behind him 
or her and trying to get through. This is especially important in light of today's quieter 
vehicles where sound from outside is significantly less audible than has been the case in 
previous generations of vehicles. The present invention is compatible with single and/or 
1 5 multiple direction information communication and display. 

It is also possible that the user might be interested in traffic conditions flowing in 
the direction contrary to the way that they are operating their vehicle, to solve the 
problem of the well- known "gaper's block" where an accident or some other incident in 
tiie opposing flow of traffic causes one's own flow of traffic to slow down. 
20 The processing means that takes the signals from the receiving means and formats 

them for display can have a number of additional functions. 

Cleariy the process of taking the information received via radio frequency or 
microwave, or satellite, or cellular phone or any of the other common wireless ti-ansfer 
means has to be converted in some way so that the user can actually understand that 
25 particular data. It can be displayed as a schematic kind of information showing a stylized 
road and stylized vehicles; it can be actual image content showing tiie view of die right- 
of-way ahead of tiie operator in a pictorial form; or it can be a combination of the two. If 
it is a pictorial of the roadway, an imaging processing operation may be perfonned on 
tiiat image content to make it more suitable for display. Such an image processing 
30 operation might include, for example, adjusting the brightness or conti-ast of the display 



WOOl/43104 PCT/USOO/33433 

-31- 

to correct for bright highlights as a result of sun or some other light source reflecting off ^ 
of the vehicles in the trafiFic flow. 

Depending upon the source of the image, noise reduction can be required to 
remove a herringbone or other kinds of obvious image noise that might be present in the 
5 image. The processing can integrate a series of images over time, to result in a different 
representation of the traffic flow that the user might find useful. Filtering operations can 
be performed to sharpen edges or accentuate certain attributes of an image especially in 
the conditions of poor weather when fog, rain or snow may otherwise obscure important 
features of the road ahead. The image processing can be geared to remove altogether 
10 artifacts such as highlights or reflections or distracting road signs from the field of view. 

Further, the image data that is being received need not be characteristic of visible 
light. It can be representative of thermal, sonar, ultrasound, or other imaging types. For 
example, the image data can originate from a thermal imager which provides information 
about the temperature of its surroundings. This processing is commonly used in 
15 applications such as night vision or night spotting scopes where infirared energy is 
converted from the invisible spectrum to the visible spectrum. This processing can be 
used in the present invention so that the user can have an image of what ahead of them in 
the roadway even in the absence of visible light based images. Given that most features 
of traffic flows that might be of interest to a user generally involve heat, infrared is a 
20 particularly useful technique to use. The range of infrared at night can far exceed the 
range shown by a user's headlights. If a person or an animal is in the road, or another 
vehicle, they tend to give offbeat in the infrared spectram which would be easily 
detectible at quite some distance and thereby give the operator of the vehicle an early 
warning as to whether there is a road obstruction or some other hazard that might effect 

25 the operation of the vehicle. 

The processing means can perform several different image processing operations 
concurrentiy or simultaneously. For instance, converting from infrared to visible light 
and performing noise reduction is a very common combination. Similarly, brightness and 
conti^st correction are generally performed at the same time. Other image processing 

30 operations are also within the scope of this invention. There is nothing that limits tiie 
class of image processing operations to those expUcitiy listed herein. Detailed 
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information on reasonable image processing steps to perform on images to make them 
more intelligible to a user or have them convey more information are readily available in 
image processing text books, such as: rr>mp..ter nranhir.s and Annlicatjons. by Foley 
and Van Dam, and Tm^p^. Processing by Gonzalez, and many other titles which are 
5 readily available from academic publishers and from professional organizations such as 
the IEEE. 

The receiving means actually can also acquire multiple data types simultaneously 
from an external source, wherein the processing means is utilized to combine them. For 
example, (1) an imSge of what is ahead in the right-of-way might be combined with 
10 textual legends indicating the distance to an obstruction or another vehicle, or (2) 

information about the location of the operator's vehicle with respect to a specific element 
being displayed. 

Estimated travel times, recommended speed, recommended side of the road to be 
on, or which lane to be in. can either be combined in the receiving means and relayed to 

1 5 the processing means, or they can be combined in the processing means as received. 

The user can select a portion, or portions of the data received for display where 
certain information might be interesting if the operator of the vehicle is making rapid 
progress. Other times, different kinds of information might be interesting when the 
vehicle is not making progress. The user's requirements can change overtime, and the 

20 present invention provides the user with the ability to select which kinds of information 
or image content are displayed. Further, the user can select to display only the data, or 
only the image information, or only parts of one or both, even though both can be 
received. 

In an alternate embodiment (e.g., see FIG. 41). the display means comprises a 
25 plurality of displays, operable to provide a plurality of selectable display modes 

comprising combinations and permutations, wherein different images can be displayed on 
each display, the same images can be displayed on each of the displays, the same data or 
different data can be displayed on the various displays, or there can be a combination 
where image content is displayed on one and other received data is displayed on another 
30 display, in any combinatibn. Means are provided for the user to select which elements 
of the received information end up on which displays. 
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In the preferred embodiment, the user configures the system to show particular 
types of information on particular physical displays by selecting from a list of 
predefmed correspondences between the information displays and the physical displays, 
user preferred correspondences between the information displays and the physical 
5 displays, and a user selectable mode screen. In one embodiment of the user selectable 
mode, the system shows on one portion of one screen a miniature view, or thumbnail 
view, of each of tiie kinds of information displays tiiat the system can generate and on 
another portion of the same display are miniature of thumbnail representations of the 
available display screens. The user uses a pointing device to click on tiie miniature 
10 information display and then clicks on the miniature available display screen, thereby 
assigning that information display to that available display screen. In an alternate 
embodiment, the user uses die pointing device to drag a representation of the thumbnail 
of the kind of information display on top of the representation of tiie available display 
screen. This dragging metfiodology is intuitive and a similar mechanism is present in 
15 conventional microcomputer operating systems. In yet anotfier embodiment, tiie user 
selects a thumbnail of a kind of information screen and tiien draws a connection line 
between that information display tiiumbnail and a thumbnail representing an available 
display screen. In a different embodiment, the user selects tiirough a keypad apparatus a 
letter or numeral corresponding to a particular type of information display tiiat tiie 
20 system can generate, and a corresponding letter or numeral corresponding to an available 
display screen to assign tiiat information display type to. In an alternative embodiment, 
die letters or numerals may be replaced by directional arrows, or a forward^ackward 
•nexf-type of selector. In anotiier embodiment, tiie means to select comprises a 'next' 
and a 'previous' button, which allows tiie user for each available display screen to 
25 sequence tiirough tiie types of information that can be displayed on tiiat screen. The 
user continues to activate either tiie next or previous button until tiie desired display 
type appears on the available display screen. One variant of tiiis embodiment replaces 
the 'previous' and 'next' buttons with a wheel control tiiat the user rotates in order to 
sequence through the available types of information displays. 
30 the information tiiat is being received by tiie system in tiie operator's vehicle can 

come from a number of different sources eitiier individually or simultaneously. For 
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example. it can receive infomation from a traffic emergency warning beacon placed on a 
roadside or wayside relaying information about the vicinity where the beacon is placed. 
This can be a permanent installation or a temporary installation. If it tends to be near an 
interchange where traffic congestion is very common it may be permanent. If it happens 
5 to be placed near where road construction is being performed, it might be a temporary 
Idnd of beacon. 

Alternatively, the data and or image content can be relayed by a transmitter on an 
emergency vehicle. The emergency vehicle may be en route to a traffic incident or it may 
be on site. This feature of the invention allows the operator of the vehicle to respond to 
10 an emergency vehicle that may be in the nearby vicinity of the operator's vehicle. Such 
response may include, for example, getting out of the way of the emergency vehicle. 

The image information and other data received by the system in the operator's 
vehicle need not come from a road side source or on board the user's or another vehicle 
that happens to be on the roadway or in the air. The information can also be provided 
15 from a central traffic monitoring location or a central data collection point. Data relayed 
from a central source may include both image and data content (for example, information 
about distances to particular exits, or travel times from exits to exits). The processing 
means can use this infonnation to display continuously updated estimated time of arrival 
to the user based on gross overall traffic conditions as reported from the central location. 
20 Similarly, this information can be derived from any localized beacons such as a fixed or 
temporary wayside beacon, or a beacon on an emergency vehicle, or from signals received 
from other vehicles or aircraft or satellites. 

In one embodiment, a central station reports travel time from a reference point in 
a route to the location of each known beacon, and also to the location of each known exit. 
25 As the user travels along the route and passes each beacon, the system in the user's 
vehicle will detect a signal strength for each beacon that gets stronger when the user's 
vehicle is within proximity of each beacon, and fades the user's vehicle from said beacon. 
In this embodiment, the user's vehicle can utilize information from a central station as a 
rough estimate of travel, times and then receive more refined or updated infonnation from 
30 roadside beacons as the user' s travels. 



3NS0OCID <WO 0il3104A1 I > 



WOOl/43104 PCT/USOO/33433 

-35- 

As shown on the flow chart of FIG. 43C, in the first step 4381. the user enters a 
desired trip itinerary into the system which may include the start location, the 
destination location, and any way-points the user expects to pass through on the user's 
route. The next step 4382 resets an elapsed time counter to zero. The next step 4383 
5 determines whether a signal is received from a central station. If such a signal is received 
from the central station, the next step 4384 is to store and retain the estimated travel time 
for each exit and to each beacon as provided to the central station within a memory of the 
system. Once this stored and retained step has completed, and also in the case of there 
being no signal received from the central station, the next step 438S is a determination to 
10 be made is whether a signal has been received from any beacon. These beacons may be 
roadside beacons or beacons located on emergency vehicles or other vehicles in close 
proximity to the user's vehicle. If there is no signal received from any beacon, then the 
next step 4386 is to update the displays in the user's vehicle responsive to the current 
information that is known by the system and the elapsed time counter. Processing then 
15 continues at step 4383 with the determination of whether there was a single received 
from the central station. 

In the event that a signal is received from a beacon, the next step 4387 is a 
determination that is made is whether the strongest signal that is being received from a 
beacon corresponds to the same beacon as was last the strongest signal. If this is not the 
20 case, this is an indication that the user has traveled out of the range of one beacon and 
into the range of a second beacon. At step 4388. the system in that situation makes note 
of tiie fact that the current closest beacon, that being the beacon witii the strongest signal 
is now considered the current beacon. After that note, and in tiie event that the strongest 
signal is from the same beacon as it was previously, tiie next step 4389 is tiie 
25 determination of whether this particular beacon has updated information. If it does have 
updated information, then at step 4390, the stored and retained estimated travel time per 
exit and beacon provided by tiie central station is tiien updated using tiie information 
from tiie current beacon. The next step 4386 is tiien to update tiie displays in tiie user's 
vehicle responsive to the current stored information and tiie elapsed time. Processing 
30 resumes again at step 4383. looking for a signal firom the central station. 



NSOOCID <WO ^0i43104A1J_> 



PCT/USOO/33433 

WO 01/43104 

-36- 

As a result, if the user's vehicle initially receives a report from a central station 
that gives it information on travel times but does not receive any other information en 
route, the elapsed time counter vnW provide the system a means to give the user a rough 
estimation of what the remaining travel time is. while the user is en route. If there is no 
5 signal received from the central station but the user does receive information from 
roadside beacons along the way. the roadside beacons provide current and up to date 
information that permits the system to then provide updated information regarding the 
total travel time. 

Video cameras are commonly available from video camera manufacturers, such as 
10 Sony and JVC and Ikegami. The transmitting means, or radio frequency transmitters, are 
commonly available through a variety of vendors, including Motorola and other 
companies tiiat specialize in radio communications and two-way radios. It is not usually 
a requirement for the system to use a two-way radio wave. A single direction transmitter 
on a beacon would be sufficient for most embodiments. 
15 In addition to the previously mentioned radio frequency and other methods of 

relay of information, information can also be conveyed to the operator's vehicle via an 
infrared receiver, or a visible light receiver, or a GPS receiver, or a cellular network 
receiver, or satellite receiver. In accordance witii the present invention, a plurality of 
simultaneous receptions via any of the above means can be received and processed. An 
20 example is simultaneously receiving remote information via a cellular network, and 

receiving local information via an infrared receiver. The processing means has a way to 
discriminate between signals tiiat arrive simultaneously. The user selects which image 
content or data information is desired, and the processing means automatically generates 
and relays video «gnals to the display (or displays) witiiout requiring the user to do 
25 anything special to receive it from one source or another. 

There are several ways to discriminate between the various signal sources that my 
be transmitting data to the present invention. The first means to discriminate relies up 
detecting which transceiver received tiie incoming signal. For example, if the system is 
receiving signals from both an infrared transceiver and a radio frequency transceiver, the 
30 system is aware, based on which input the signals arrive in. what the signal source of that 
particular signal was. An additional means to determine or discriminate the source of the 
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signal is an ED code that is transmitted along with the message or data or image from each 
transmitter that identifies the source and also the type of information being conveyed. 
For example, the ID code may indicate that the source is from a helicopter hovering over 
the traffic area, from a roof-top camera, from a camera on a particular billboard, from an 
5 image gathering source in a mobile or emergency vehicle, or from a central station. The 
types of information that might be conveyed include for example, video information, 
image information, congestion information, traffic pattern infomiation, travel time 
information, road condition information, the sensor's location, weather condition 
information, construction alerts, a warning of road hazards or of disabled vehicles or 
10 locations where vehicles such as a farm vehicle might be moving slowly. The sources and 
data types permit the system to intelligentiy select the best source and type of 
information for the user based on the desired image display. For example, traffic 
congestion information that is relayed from a mobile vehicle in the immediate proximity 
of tiie user is more likely to be accurate than the same type of information available from 
15 a central reporting center, and therefore, the system could intelligentiy permit on-site or 
near-by information to override information provided by a central information center. In 
another example, the information provided from a mobile vehicle, such as a view of traffic 
congestion or information about road hazards, might be overridden by the similar type of 
information being transmitted from an emergency vehicle in the area. As yet another 
20 example of the intelligence of the system, the system may elect to override purely 
informational displays requested by the user, such as travel times or congestion 
information, and display road hazard information or adverse weather condition 
information that require the more immediate attention of the user. 

Examples of how to automatically generate and relay signals is covered 
25 hereinabove, such as in the description of how the user selects a particular kind of 
information display to appear on an available physical display. 

The system can have a number of video generation circuits such as video 
adapters, that are commonly used for personal computing. Examples are VGA devices 
manufactured by ATI or Matrox, or S3. All of them provide graphic chips that are 
30 specifically designed to generate high resolution displays (examples of tiiese are available 
on the World Wide Web). In addition, there are a number of older display generation 
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devices which will work on lower resolutions that are probably more typical of what 
would be in a moving vehicle. 

In a preferred embodiment, the display that the user observes is actually located 
in the user's vehicle. It may be located so that the operator of the vehicle can see it, or so 
5 that another occupant of the operator's vehicle is able to monitor the display. In 

jurisdictions where a operator->dsible display is not permitted by law, having the display 
visible by a co-pilot is the advised method of operation. Alternatively, or additionally, 
audio or speech can be utilized instead of a visible display. In an alternate embodiment, 
the display may actually be external to the user's vehicle such as a roadside billboard 
10 video display, or a system of indicator lights (or display elements (e.g., character, live)) 
either in the roadway, above the roadway, or to the side of the roadway giving 
information, instead of or in addition to a display within the user vehicle. Roadside 
displays may also include textual displays that would relay information for the user to 
read as the user is travelling. 
15 In accordance with one embodiment of the present invention, the display is 

incorporated in a central monitoring location where that information is used by one or 
more users at the central monitoring location, such as to determine, for example, 
emergency response to traffic conditions, responses to temporary congestion on the 
basis of construction, or normal and abnormal traffic flows. 
20 In a preferred embodiment, the invention includes an image or other kind of 

sensing means (that is supported in one of a plurality of positions with respect to the 
operator's vehicle) coupled to a processing means which provides a display output 
responsive to information received from the sensing means for display to the operator. 

In one embodiment, a camera, or other image sensing apparatus, is supported on a 
25 vehicle providing a view of traffic around the vehicle, and relays that view via the 
processing means to a display for viewing by the operator or other person of that 
vehicle. This allows the operator to have a viewpoint of traffic conditions different from 
the operator's normal perspective of sitting within the vehicle. For example, the 
operator's normal perspective might be obstructed from seeing something that is ahead of 
30 the operator in the traffic flow, because a large truck was in front of the operator's 

vehicle. Utilizing tiie present invention, the operator can see around or over the truck. 
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thereby giving the operator a visual representation of what exists beyond the immediate 
obstruction. 

The sensing means or camera can be on a fixed mount. Alternatively, the support 
can be a retractable system that allows the camera to be selectively elevated sufficiently 
5 high above the operator's vehicle to be able to see over large obstructions, and/or it can 
also be extended out fi-om either side of the operator's vehicle thus allowing the operator 
to see around a traffic obstruction that is viewable from a different angle. The support 
can be a boom arm that allows a plurality of these motions to be accomplished by lifting 
the sensor above the top of the vehicle and independently or simultaneously translating 
10 or shifting off to one side. 

In one embodiment, the retractable support comprises a hydraulically actuated 
means to lift the camera up. In an alternate embodiment, it can be an electric motor 
driving a mechanism similar to a retractable radio antenna for a car. In other 
embodiments, it can be implemented with a scissors lift device, telescoping rod 
15 arrangement, a single arm boom, or multiple arm jointed boom similar in geometry to 
what you might find on a snorkel fire engine. 

Any type of support mechanism that supports the sensors or cameras is 
compatible with the present invention. Retractable supports are commercially available 
from many sources. Examples include all major automobile manufacturers and parts 
20 suppliers that supply retractable antennas for major automobile manufactures. On an 
industrial scale, there are also numerous commercial companies that supply the hydraulic 
lifts that lift up microwave dishes for tiie mini-cam vans, and have a similar product on a 
smaller scale that can be used for smaller vehicles. 

One such support comprises a plurality of nested cylindrical tubes of sequential 
25 decreasing diameter, having alternately swaged ends to prevent the tubes from separating 
from one another, said tubes configured in a telescoping arrangement, with an electric 
motor (such as a 12-volt motor commonly used to operate automobile accessories) that 
drives a take-up spool onto which a semi-rigid support is coiled with one end of said 
semi-rigid support affixed to the smallest cylindrical tube. As the motor operates in a 
30 forward direction, the semi-rigid support is unwound from the spool and thrust into the 
telescoping arrangement of tubes forcing said tubes to extend telescopically. When the 



iSOOCtO- <WO P143104A1 J_> 



PCT/USOO/33433 

WO 01/43104 

-40- 

motor operates in the reverse direction, said support is withdrawn from the tubes and 
wound around the spool, thus forcing said tubes to collapse and shorten telescopically 
into one another. 

In a preferred embodiment, the support means also includes a means to allow the 
5 camera or sensing device to be further altered in position and orientation, such as the 
ability to pan the sensing means from left to right to allow it to see (e.g., sense, detect, 
perceive) information to one side or the other of the vehicle; to tilt the sensing means up 
or down, allowing it to sense information from close to the vehicle to far away from the 
vehicle, or perhaps even an aerial view from above the vehicle. The support also has the 
10 ability to roll the camera from side to side, either to present a different kind of image via 
the sensing means or to compensate for any motion that might be imparted by the 
support means. In an additional embodiment, a zoom mechanism allows the camera or 
sensing means to frame and focus on an area of interest responsive to the operator of the 
vehicle. 

1 5 The sensing means can relay data to the processing means via a wired or wireless 

technology or via fiber optic cable. Data can also be conveyed using the support as an 
electrical means to communicate. Alternatively, a transmitter in the sensing means relays 
data down to a receiving means actually on the vehicle. Data transmission can be infrared 
orRF or microwave or visible light or any of the transmission technologies that have 
20 been previously discussed herein. Specific examples of fiber-optic cables include cable 
produced by Kippon Sheet Glass of Japan, or Owens-Coming Corporation, which is a 
U.S. Company. Types of wired conveyance include cables of coaxial or other variants 
that are produced by Belden Corporation, a US Company. The most likely common 
variant is a standard 75 Ohm coaxial cable as used in video distribution. Infrared 
25 transmitters and receivers are available in discrete form through such vendors as 

RadioShack, as well as from a number of other vendors or manufacturers, such as Jameco. 
Digikey. and other electronic parts suppliers. RF, radio-frequency, transmitters and 
receivers are as already discussed earlier herein, such as from Motorola, Qualcomm, Sony 
Corporation, Analog Devices, etc. These are all vendors that produce products that are 
30 designed for use in wireless transmission. 
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In a preferred embodiment, the sensing means detects visible light images using 
for example, a camera or CCD array or vidicon tube. All of these technologies are 
capable of producing signals representative of a visible light image. This visible light 
image may require some processing to ideally present the information to the user. For 

5 example, a selected portion of the image might be of interest so the user can electronically 
zoom in on a section of the image. In a similar fashion the user can apply brightness or 
contrast correction to make out fine detail, or filtering to bring out edges or resolve 
differences between elements of the image. 

Some image sensing means are more sensitive than otiiers to noise induced by tiie 

10 operation of tiie vehicle itself of sources of interference in or around the vehicle. 

Performing noise reducing processing on the data from tiie sensing means can improve the 
displayed image. 

Some image sensing means, particularly CCD arrays, tend to have sensitivity in 
the infrared regions of tiie spectrum. This is usefiil from tiie standpoint of a preferred 
15 embodiment of tiie invention in tiiat it allows the system to see heat signatures of 

vehicles or persons or other elements representative of traffic or human beings around the 
user's vehicle. However, since tiie average person cannot see in infrared, the image 
received by tiie CCD array is processed and converted to provide a perceivable visual 
display representative of a visible light equivalent so tiiat tiie user can actually observe 

20 tiie infrared data. 

Multiple of these image processing operations can be performed simultaneously 
to meet tfie user's needs. In a preferred embodiment, tiie user is be able to select any or 
all of them simultaneously. 

Where visible and infrared sensing is desired, multiple-function sensors or 
25 multiple separate sensors can be utilized. 

The sensing means is not limited to sensing just image content; tiiere can also or 
alternatively be a data source witiiin view of the sensing means. For example, if the 
sensing means comprises a visible light image sensor or camera, it can receive a coded 
transmission via visible light sent from a roadside beacon. Information content can be 
30 relayed via a flashing strobe off" an emergoicy veWcle or via a roadside sign. The flashes 
are be sensed by tiie camera and information is extracted by the processing means for 
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further formatting and display for the user. This is analogous to using a signal light to 
communicate via Morse code between ships at sea in World War II. Obviously with 
electronics and electronic control of both transmission and reception, one can transmit a 
significant amount of information. In one embodiment of the present invention, the 
5 information thus transmitted includes information about an obstruction in a road, 

detours, recommended alternate paths for travel, estimated time to a particular exit, travel 
time between points, etc. 

In a preferred embodiment, the processing means selectively chooses some of tiiis 
information, under control of tiie user, to display selected portions of it on a screen. For 
10 example, the user might be interested in a detour or possible alternate paths, or might be 
particularly interested in arrival time at a destination. 

The processing means does not need to convert all the data that is received or 
sensed into a form suitable for display. For example, in tiie case of an infrared sensed 
image of tiie traffic representative around tiie user's vehicle, tiie system might selectively 
1 5 choose to ignore extremely bright heat signatures of exhausts of vehicles, tiiereby 

performing a tiireshold function where received infrared signals above a certain threshold 
are ignored or not processed and not conveyed to the user, as they contwn irrelevant 
information. 

In another preferred embodiment, where the support means is mounted to a 
20 vehicle, its deployment is responsive to tiie vehicle' s speed sensing input such that the 
support will not deploy unless tiie vehicle is moving below a preset speed tiireshold. 
The retractable support can also be set to deploy (to elevate) tiie camera only if when the 
vehicle is slower than a reference speed or stopped. This would function as an additional 
safety feature in that the operator would not be able to deploy the camera or be 
25 distracted by a display of traffic conditions around the operator's vehicle unless the 
vehicle is already slowed or stopped as a result of congestion and traffic. 

In another embodiment, the support is sufficientiy strong to allow deployment 
and operation of the camera even when the vehicle is moving at higher speeds. 

The position and elevation of the deployed camera can also be controlled as 
30 related to tiie speed of the vehicle. For example, it might be desirable to deploy a camera 
at a height of several feet above the vehicle if the vehicle is moving relatively slowly. 
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where it may be desirable to deploy the camera at a height of ten or more feet above the 
vehicle if the vehicle is moving in a faster fashion, or vice-versa, or both, thereby 
permitting the sensing means to see both closer and further ahead in the flow of traffic 
from the vehicle's position. 
5 In an alternate embodiment, the support means deployment is responsive to 

other factors such as relative wind speed. The processing determines the support means 
deployment based upon factors such as the vehicle's speed and the direction of the 
ambient wind. This would, for example, prevent injury to the apparatus as a result of 
deploying on an otherwise motionless vehicle in a heavy cross-wind. 
10 In one embodiment, the retractable support allows the camera to retract down to 

a rest position which is still an operable position for the system. For example, the 
sensing means can be retracted to a position that is still capable of looking forward or to 
one side of the vehicle. In another embodiment, the retracting means may actually retract 
the sensing means within the vehicle thereby protecting it from theft or vandalism when 

IS not in operation. 

In another embodiment, the retractable support is under direct control of the 
operator thus allowing the operator to make decisions about whether or not to deploy 
the imaging sensor, and where to deploy the imaging sensor, relative to the vehicle. In a 
preferred embodiment the methods of controlling the deployment of the retractable 
20 support are combined to provide the system with multiple modes of operation, wherein 
the system is responsive to vehicle speed and relative wind speed in one mode, and is 
also responsive to a user control in another mode, and where the operator can override 
either or both the vehicle speed and wind speed sensing apparatus in yet another mode. 

In a preferred embodiment, the retractable support is affixed to the user's vehicle. 
25 In an alternate or additional embodiment, the retractable support that positions the 
sensing means is independent of the vehicle and is affixed wth respect to a fixed 
structure such as a garage, a parking structure or a roadside fixture. The support for the 
sensing means can include any wayside structure such as a light house, a street light, a 
marker buoy, an exit or entrance sign, some other informational sign, a toll booth, a fixed 
30 structure adjacent to the right-of-way, a moveable platform (moveable with respect to 
the right-of-way), a crossing signal for railroad tracks, a power pole or other adjacent 
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utility poles providing a vantage with sufficient altitude to be of use. with sufficient 
support structure such as attached to concrete (or other construction) of lane dividers, 
weighted down, aircraft based, underground based, or other means not expressly 
specified here. Regardless of where the actual support is located, the optional functions 
5 of being able to pan the sensing means from side to side, tilting the sensing means, rolling 
the sensing means, and/or zooming the sensing means would apply in all of the mounting 
variations. Similarly the operations of pan, tilt, roll and. zoom can be simultaneously 
acted upon the sensing means in order to orient the sensing means to be the most 
advantageous for the operator of the vehicle. 
10 In one embodiment, the sensing means is a camera with either a fixed focus lens, 

or a variable focus lens supporting diflferent depths of field. In a preferred embodiment, 
the camera has an automatic exposure correction system such as commonly available on 
consumer camcorders and similar products where tiie lens opening is automatically 
adjusted to accommodate for ambient light and other lighting conditions. The same 
15 exposure technology can be applied in an analogous fashion for an infrared sensing 
apparatus to automatically adjust for the total amount of signal received in an infrared 
spectrum. 

In anotiier embodiment, the sensing means has a plurality of sensors, of 
homogeneous or heterogeneous types. For example, a sensor comprising botii an optical 
20 sensing apparatus (such as a television camera) and a CCD array (for sensing infrared 
imaging) may be part of tfie same sensing means. Alternatively, one CCD array witii 
fiinctionality in both areas of the spectrum could be used. 

In another embodiment a plurality of homogenous sensors such as an array of 
CCD arrays (or an array of oti»er imaging sensors) looking in the same direction or in 
25 slightiy different directions provide a wider field of view without requiring panning or 
tilting operations by the user. 

In an alternate embodiment, multiple sensors are used to produce a stereoscopic 
image of representative of traffic around tiie operator's vehicle, tiius allowing tiie 
operator to use deptii perception on suitable displays, such as heads up displays 
30 supporting depth perception (or with suitable display processing on a conventional 
display), to make assessments about the relative positions of vehicles thus displayed. 
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The processing can combine the stereo images or multi-viewpoint images thus 
obtained by a plurality of sensors to create and modify the display presentation. For 
example, processing can be used to add highlighting or assigning of color to objects that 
are closer that might be of more significance to the operator of the vehicle. 
5 The electronic visualization system in accordance with the present invention is 

used to present an electronic representation of traffic flow in and around location of a 
vehicle. This is accomplished by having a sensing means designed to detect vehicles from 
an elevated perspective position relative to the operator's vehicle. This allows the 
sensing means to detect vehicles that would ordinarily not be visible to the operator of 
10 the vehicle from the typical operator's position. By elevating a sensing means above the 
operator's vehicle, a new perspective is gained in a bird's eye view of the traffic 
situation, and tiiat content from the vehicle sensing means is communicated to a 
processing means in the user's vehicle which the processing creates for the user a 
visualization representative of traffic patterns around the vehicle. This allows the user, 
15 for example, to have a visualization of a traffic blockage not located immediately next to 
his or her vehicle and tiius provides advance warning to that operator of tiie vehicle of tiie 
need to reposition within the traffic flow (e.g.. to the left or right lane) in order to avoid 
that blockage. 

The data that is sensed by the vehicle sensing means can be representative of 
20 video data, individual still images, or other sensor data signals, conveyed to the 

processing means. The processing means can provide for display of the data directiy as 
video and it can perform processing such as feature detection and content recognition in 
Older to generate tiie electronic visualization display representative of the traffic. 

The data from the overhead perspective can come from a sensing means actually 
25 attached to the vehicle via some support (which can be a retiractable support), and/or it 
can come from a sensing means attached to a different vehicle or vehicle that is not part 
of tiie traffic flow, such as a helicopter hovering over the ti^ffic flow, or a sensor attached 
to some fixed or moveable structure in the vicinity of tiie tt-affic flow. 

The data from the sensing means is communicated to the processing means of tiie 
30 user's vehicle. There can be multiple sensing means in use simultaneously, for example a 
sensing means present on support affixed to tiie user's vehicle, anotiier sensing means 
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affixedtoaroadside Sign, another sensingmeans affixed toabuiW^^^ 

another sensing means inahelicopter hovering over the area, and another sensingmeans 

attached to another vehicle in the vicinity of the operator's vehicle. Each sensing means 

can be stationary or mobile. 
5 The processor accept i.n.g. or daU «»..en, from of diir««.. sources, or 

. user's sel«=tion of !hese sources. «.d provides for combining them .s required or 
o*e™ise in«grati„g «.e infcrmation for a coherent display .o .he us«. This display can 
involve overlays and/or picmre-in-picure visual pre^n«tions.boA well kno«nm«.e 

^ in video «.d image presentadon. AlKmaUvely. .he images received from mutople 
10 sensors can be simuluneously displayed as a mosaic of smaUer images, allowing *. u^ 
to select a view of any one of ttem for larger display. 

to a preferred embodiment, the processing means can be programmed or 
controlled by the user to sequence through a plurality of different modes of display. For 
example, the user can select cycling at a user con»o.led mte be^»e«. differ«.t vews 
15 available to the user' s vehicle from other sensing me«.s. or perhaps from muluple 
imaging sensors that are on the user's vehicle itself The cycling operation connnuea 
undl the user selects a particular view to suy with, and it periodically updates ti,e 

available based on the availability of image vehicle sensing source, nearby 
becoming available, and odier sources becoming unavailable. 

in an alternative embodiment, a system on anoti.er occupant' s vehicle can 
,™„mit image content or d«a to ott,er vehicles nearby inatrafficflow.includi„gU.e 

opemtor-s vehicle, which receives data from ti.e ottier vehicle's sensing means va 
wireless transmission. This operation conveys image dati. from another vehicle to tire 
user via one tiansmission "hop". In »iJogous fashion ti.e user's vehicle may also 
25 include a tiansmitter that relays information received and/or se»«d (via a sensor local to 
fte user' s vehicle) to oti.er vehicles in me tn.Hic flow. The im,ention thus gr«.ti> 
opemtor of a vehicle a view of traffic significantiy forward of his or her position based on 
having imag« relayed ti-rough multiple "hops" via a multiple sensing and relaymg me«,s 
f^om otfier vehicles or from other fixed locations. As . result this invention implements a 
30 cooperativenetworkwhereimagedattbeyondtheviewofauserinthetraffioflowcan 

be relayed from quite some distimce from ti.e user's vehicle via a series of "hops" e.Aer 
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from mobile platforms such as other vehicles in the traffic or through relatively 
stationary 

platforms such as buildings or roadside fixtures. 

In supporting the function of relaying data within the traffic flow to other 
5 compatible systems, the processing means provides additional functions, such as data 
compression, data encryption, and or data origination tagging. Data compression is used 
to reduce the amount of data transmitted to a reasonable amount to send via any of the 
previously mentioned transmission and reception means (e.g., radio frequency, infrared, 
and so forth). Data encryption can be used if the image content was being sensed for a 

10 purpose other than immediate supply to other operators of vehicles in the area (for 
example, relayed to a central monitoring area where the information is used and 
subsequently resold in some fashion). 

Data origination tagging is simply a way of identifying the origin of the data being 
transmitted from a given vehicle. For example, if a vehicle operator is using the present 

15 invention to transmit image data that is being acquired by an image sensing means 

mounted to that operator's vehicle, it would be tagged with an identification indicating 
which vehicle (or that vehicle's location) is sending that data and its absolute location as 
determined by position relative to local way points, fixtures or landmarks, or relative to 
the global positioning system. Data origination tagging allows other receivers in the traffic 

20 flow to use tag information to identify where in the traffic flow this transmission 
originates from, and if it is relevant for their own respective use. 

Other data origination information that can be transmitted includes the relative 
and absolute direction of flow. For example, if a given user's vehicle is travelling in a 
northbound lane, other northbound vehicles might be veiy interested in receiving 

25 transmissions from a northbound vehicle ahead of them, while they would obviously 
have no interest in receiving information from northbound vehicles that are somewhere 
south of them and therefore behind them. Southbound vehicles would likewise probably 
have little interest in information transmitted from northbound vehicles regardless of their 
relative location to the southbound vehicle. Data origination tagging is a powerful feature 

30 because it allows the user select from the plurality of images and data that arrive at the 
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operator*s vehicle from a variety of dtfTerent sources, and to collapse those data sources 
down into a manageable number for subsequent processing and display. 

In a preferred embodiment of this aspect of the present invention, information is 
transmitted from an operator's vehicle in a broadcast fashion thus allowing other vehicles 
5 to selectively discriminate between the totality of signals being received to determine 
whether any particular transmission is relevant. Alternatively, the transmission from a 
visualization or early warning traffic system present in the vehicle can be made highly 
directional, such as a hooded light source or directional communication beacon directed to 
a particular direction from the user's vehicle, or to particular reception sites located with 

10 respect to the operator's vehicle (for example, a particular roadside collection point or 
central monitoring location). In the case of a sailboat incorporating this traffic 
visualization system, the image data can be communicated in a direction downwind to 
other craft such as those that would be required by the rules of navigation on the water to 
yield to the sailboat, if they are equipped with similar systems, thus allowing the 

15 operators of the other vehicles to take action to avoid collision or otherwise operate their 
vehicles in a safer manner. 

In yet another embodiment, data is communicated to the user's vehicle and 
processed to provide for the display to the user of a schematic view of traffic conditions 
surrounding the operator's vehicle. This can be accomplished by using (1) a number of 

20 data or image content sensors located at different positions with respect to the right of 
way and (2) a central (or distributed) collection apparatus comprising processing and 
transmission means that use the data collected by the various sensors and present it in a 
broadcast fashion such that vehicles within the traffic flow could receive it and format it 
in a display suitable for the user. 

25 For example, a roadway containing sensors detecting positions of vehicles can 

relay that data to a central location where that data is assembled into an instantaneous 
mapping of traffic on that roadway. That mapping is then transmitted or broadcast to 
the vehicle safety early warning systems, which use the position of the operator's 
vehicle relative to roadside beacons, or landmarks, or the global positioning system, or 

30 another positioning system, or sensing the ID's of the roadside sensors, resulting in a 
comprehensi ve display of nearby traffic at whatever resolution the user might require, 
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integrated in with a methodology to allow the user to select a destination for a ^ven 
roadway, such as a particular highway exit. 

In accordance with the present invention, the user is given a comprehensive 
display indication (representative of the traffic flow around the vehicle) of the amount of 
5 traffic that would have to be navigated in order to reach the destination, and any road 
hazards or other obstructions that might be sensed by the network of sensors in and 
around the highway. The system can even instruct the user either via display or audio, 
such as that the middle lane of three lanes of traffic happens to be moving the fastest in 
the area where the user is located, and that when safe to do so, the user should effect a 

10 lane change to be in the middle lane to make the best forward progress. 

This type of system can also be implemented without a separate network of road 
sensors, such as by using the previously disclosed imaging sensors on roadsides or 
structures or mobile platforms to collect the same data representative of the overall 
traffic patterns. Similarly the processing means on board the user's vehicles can distill 

1 S those details that are relevant to the particular user from the data being broadcast 
regarding the instantaneous traffic flow. 

In another alternate embodiment, the information is relayed locally or from a 
central location about non-traffic factors that may affect traffic but aren't directly caused 
by traffic. For example, local flooding conditions, local icing conditions on bridges, or 

20 other very local weather patterns might be relayed to the user's vehicle from a sensing 
apparatus located roadside or within the right-of-way thus giving the user advance 
warning of those hazardous conditions. For example, sensors embedded in an overpass 
may detect whether or not the road surface has sufficient moisture on it and is 
sufficiently cold to have frozen over. This information can be broadcast from a localized 

25 emergency beacon thereby warning travellers about to arrive at that overpass to slow 
down. Additionally, information can be relayed about toll plaza booth availability for 
toll roads where given lanes of traffic are spread out over a plurality of toll booths. 
There may be an indication that one or more booths are inoperative at the given time or 
have an unusual traffic pattern associated with them. That information allows the user to 

30 select a rnore expedient toll booth lane to be in. Having that knowledge in advance before 
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arriving at the toll plaza would make the transit of the toll plaza more efficient for the 
user. 

In accordance with the present invention, the display need not be limited to a 
display actually present in an operator's vehicle. The display can also or alternatively be 

5 located separate from any user's vehicle, such as housed in a structure, for example, for 
study of local traffic conditions by police or news media, or to analyze if a municipality 
needs to affect a change in a local right-of-way. It might be used in an alternate 
embodiment as part of a central monitoring station where traffic patterns over all are 
analyzed and reported. In the example of a roadside display such as a billboard type 

10 display (or other array of light display), the sensing means can be local to the display 

(such as one mounted on or affixed to that structure) tliereby providing a fixed local view. 
Alternatively, the sensing means for such a fixed roadside display can be located some 
distance from the actual display itself, giving user's advance warning of an area 
significantly distant. 

15 The case of a fixed structure blocking direct view of the path of the road is 

commonly evidenced by expressways winding through urban areas. Those blocking 
buildings prove to be ideal mounting points for fixed cameras to observe traffic patterns 
in that they would allow a direct view (to users approaching the buildings) of 
obstructions that may be beyond them. 

20 In a preferred embodiment, the structure supporting an image sensing means is 

isolated from shock or vibration. If the sensing means is actually affixed via a fixed or 
retractable support on a vehicle, the preferred embodiment is to have the assembly shock 
mounted in a fashion that reduces the amount of vehicle vibration coupled by the sensor. 
Alternatively, using techniques that are well known in the art of consumer camcorders, 

25 jitter reduction may be performed by the processing means to reduce the appearance of 
vibration in a sensed image. For installation such as at the top of a mast of a sailboat, the 
mounting for the sensing means may also include gyroscopic stabilization. This might 
also be used if the sensing means is located in a mobile platfonn such as a helicopter. 
There is no technical reason other than cost that would prohibit this from being used in a 

30 user's vehicle, which would also allow a veiy stable image to be provided to the user. 
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The ability to use a retractable support stiucture located on the operator's 
vehicle, which allows an image sensing means to be positioned at a height above the 
operator's vehicle, enables (for example) a safer traffic passing operation where the 
vehicle operator would be able to look beyond the vehicle directly in front of his or her 
5 vehicle to determine whether or not it is safe to pass. 

In a preferred embodiment, the vehicle ahead that is blocking the operator's vision 
would have its own sensing apparatus and transmitter and would relay directly an image 
of what is located in front of it back to the operator cashing to pass. This method of 
relay might be via a broadcast transmitter, or in a alternate embodiment a modulated light 
10 source such as an LED or an infrared emitter transmitting data backwards from the 
blocking vehicle to the operator's vehicle. 

This process of relaying image or data content backwards through the traffic flow 
from one vehicle to the next is another important aspect of the present invention, because 
it allows vehicles further back in the traffic flow to have the benefit of image and data 
15 content representative of traffic conditions in front of them, thereby giving them 

advanced warning of road blockages and/or hazards that would otherwise be invisible to 
the operator's of those vehicles. 

In another networked vehicle embodiment, a plurality of vehicles (each equipped 
with image sensors and processing means and transceivers) have the ability to 
20 communicate images and data via the transceivers between one another. The group of 
vehicles each have a relative orientation within the traffic flow and also a relative position 
with respect to one another. The processing means further comprises an election system 
that allows each vehicle operator to select, for display presentation, data comprising 
either images or data content from other vehicles in a selected direction from the user's 
25 vehicle, or from the imaging sensor that is part of the operator's own vehicle. 

While the description herein has concentrated on terrestrial applications, the 
present invention is also applicable to aircraft and space craft. Some applications of 
remote sensing apparatus that is retractable also applies to space craft, such as to judge 
docking distances or clearances. In accordance ^th the present invention, in a geneve 
30 context, a plurality of cameras or sensors with transceiving means send information back 
and forth to each other and receive data from the closest beacon or camera position. 
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In accordance with another embodiment of tfie present invention, another 
application of this technology is using it as a black box to capture telemetry of a vehicle. 
For example, there are situations where police squad cars or cruisers have cameras 
mounted to view (out the front windshield) the activities involved in a chase or an 

S apprehension or an arrest. These signals can be processed and used in accordance with 
the present invention. In a similar vein, the present invention can be used with other 
types of public or commercial vehicles, for example, cameras/sensors/transmitters could 
be put on taxicabs or on the cabs of trucks for a commercial hauler to allow them to 
capture video information about the operation of the vehicle. The captured video can be 

10 transmitted and/or recorded to a memory of some sort continuously while the vehicle is 
in operation. The memory can record over the same sections over and over, except in the 
event of an accident, thereby providing 10 to 90 minutes (design option) of context to 
any accident investigators that might happen upon the scene. So even if the operator of 
the vehicle is incapacitated by the accident, if the device survives, then the company or 

IS next of kin might have an indication of what happened. This can be combined with other 
signals and telemetry from the vehicle, such as pressure in the brake lines, manifold 
pressure in the engine, whether or not the engine was actually running at the time of 
impact, whether there was a steering input being applied, whether the vehicle was in 
panic braking, or any of those kinds of factual information. From a manufacturing, 

20 safety, public interest, and insurance standpoint, this type of technology is very useful 
and can be employed as a logical extension of what already exists in on-board computers 
that arc monitoring engines and systems in cars, adding telemetry and external inputs, 
such as video and/or sensing apparatus. 

From the foregoing, it will be observed that numerous variations and 

25 modifications may be effected without departing from the spirit and scope of the 

invention. It is to be understood that no limitation with respect to the specific apparatus 
illustrated herein is intended or should be inferred. It is, of course, intended to cover by 
the appended claims all such modifications as fall within the scope of the claims. 
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WHAT IS CLAIMED IS: 

1. An electronic traffic visualization system comprising: 

a receiver for receiving content signals representative of surrounding traffic 

conditions from at least one location; 
5 a processing subsystem for processing the received content signals to foimat the 

content signals for display to a user, and 

a display apparatus for providing a presentation of the surrounding conditions 

representative of the traffic to the user responsive to the received and processed content 

signals. 

10 2. The system as in claim 1 , wherein the traffic conditions comprise traffic 
conditions located forward of the position of a vehicle. 

3 The system as in claim 1 , wherein the content signals are comprised of both data 
content and image content signals representative of traffic, wherdn the processing 
additionally comprises combining both the data and the image content signals into a 
15 combined format suitable for a corresponding combined display to the user. 

4. The system as in claim 1 , wherein the processing selectively translates image 
content outside the range of humanly perceptible wavelengths into image content that can 
be displayed entirely within the range of humanly perceptible wavelengths. 

5. The system as in claim 1, wherein the content signals representative of traffic are 
20 both received from an external source and locally sensed by at least one sensor, and 

wherein the processing additionally comprises processing both the received 
external and locally sensed content signals to provide for a display presentation to the 
user corresponding to both the content signals. 

6. The system as in claim 1 , further comprising a transmitter for transmitting at least 
25 one of the received content signals and the formatted content signals to a location remote 

from the vehicle. 

7. The system as in claim 6. wherein the transmitting further comprises broadcasting 
content signals. 

8. The system as in claim 1, wherein the processing comprises at least one of the 
30 operations of content combining, content discrimination, content integration, content 

layering, edge detection, brightness correction, contrast correction, noise reduction. 
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integration over time, filtering, artifact removal, spectrum conversion, data compression, 
data encryption, and data origination tagging. 

9. The system as in claim 8, wherein the processing comprises a plurality of the 
operations. 

5 10. The system as in claim I, further comprising: 

a sensor apparatus for sensing content signals representative of surrounding 
traffic conditions; 

positioning means for positioning.said sensor apparatus in a selectable position 
relative to a vehicle; and 
10 wherein the processor subsystem is additionally responsive to the sensed content 

signals. 

11. The system as in claim 10, further comprising at least one of vehicle speed 
sensing means and relative speed sensing means, and wherein the positioning means is 
responsive to at least one of the vehicle speed sensing means and the relative speed 

IS sensing means. 

12. The system as in claim 1, wherein the receiver further couples and decodes 
' transmitted signals from at least one sensor. 

13. The system as in claim 12, wherein at least one sensor is external to and not 
attached to a user's vehicle. 

20 14. The system as in claim 12, wherein there are a plurality of sensors, wherein each 
of the sensors outputs locally sensed content signals. 

15. The system as in claim 14, wherein at least one of the plurality of sensors are 
mounted to at least one fixed structure. 

16. The system as in claim 14, wherein signals from the plurality of sensors are 
2S broadcast as a subscription service, wherein each subscriber has an enabled receiver, a 

processor, and a display subsystem; and 

wherein the content signals are broadcast to the subscribers. 

17. The system as in claim I, further comprising: 

control logic for selecting which of the content signals vail be selected and 
30 displayed. 
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18. The system as in claim 17, wherein the control logic is responsive to position 
data associated with the user viewing the display presentation. 

19. The system as in claim 14, wherein each of the plurality of sensors provides 
content signals representative of surrounding sensed conditions, and associated 

5 origination tagging data representative of the position of the respective sensor. 

20. The system as in claim 19, wherein the position is at least one of sensor location 
identification, absolute location, vehicle identification, relative direction of flow, and 
absolute direction of flow. 

21. The system as in claim 1, wherein the receiver, processing subsystem, and 
10 display apparatus are all within a vehicle. 

22. The system as in claim 1, wherein there are a plurality of vehicles each 
comprising a respective receiver, processing subsystem, and display apparatus. 

23. The system as in claim 6, further comprising the vehicle, wherein the vehicle is 
comprised of the receiver, processing subsystem, display apparatus, and the transmitter. 

15 24. The system as in claim 21, wherein there are a plurality of S£ud vehicles, wherein 
each of said vehicles is selectively responsive to origination tagging data for selecting 
which of the respective content signals to process and display. 

25. The system as in claim 24, wherein each of said vehicles has associated position 
data; 

20 wherein each of said vehicles is further responsive to the associated position data 

for each of said respective vehicles in selecting which of the respective content signals to 
process and display. 

26. The system as in claim 21, wherein the vehicle has associated position data, 
wherein there are a plurality of broadcast content signals, wherein the selection of which 

25 of the content signals to process and display is selected for traffic conditions located 
forward of the respective vehicle. 

27. The system in claim 26, wherein there are a plurality of said vehicles each moving 
in a respective direction within a traffic flow, wherein at least one of the vehicles provide 
for relaying content signals in a direction back through traffic flow relative to the 

30 respective vehicles' direction. 
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28. The system as in claim 1, wherein the display apparatus is a roadside display 
device alongside a road, viewable from along the road. 

29. The system as in claim 1» further comprising: 

sensing means for sensing content data representative of traffic conditions; 
5 wherein the processing means processes the content data to convert the sensed 

content data into a format suitable for at least one of transmission and display; and 
communication means for communicating the content data, responsive to the 

processing means, for providing at least one of a transmission of and a reception of 

processed content data. 
10 30. The system as in claim 10, further comprising support means having a first end 

and a second end, wherein the first end is coupled to the vehicle and the second end is 

positionable to extend beyond the vehicle, for supporting the sensing apparatus in a 

plurality of positions relative to the vehicle. 

3 1 . The system as in claim 29, wherein the communications is accomplished via at 
15 least one of wires, optical fiber, optical conduit, radio frequency waves, microwaves, 

infrared light waves, satellite communications, cellular communications, and visible light 
waves. 

32. The system as in claim 1 , wherein said display presentation is comprised of at 
least a visual presentation, wherein the visual presentation operates in one of a plurality 

20 of modes of visualization and view, comprising at least one of an actual overhead view, 
an actual perspective view, an actual side view, a schematic overhead view, a schematic 
perspective ^ew, a schematic side view, a virtual overhead view, a virtual perspective 
view, a virtual side view, a virtual right-of-way flyover view, a virtual radar view, an 
informational view, a navigational view, and a warning view. 

25 33. The system as in claim 32, wherein the processing additionally auto-cycles 
between selected ones of the plurality of modes of visualization. 

34. The system as in claim 30, wherein the support means additionally functions as 
an antenna. 

35. The system as in claim 14, wherein selected ones of the plurality of additional 
30 sensors are affixed to other vehicles. 
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36. The system as in claim 30, additionally comprising a user deployment actuator, 
and wherein the support means provides a retractable support that selectively deploys 
responsive to the user deployment actuator. 

37. The system as in claim I, wherein the display presentation is one of an audio 
S presentation, a visual presentation, and a combined audio and visual presentation. 

38. The system as in clmm 1, further comprising: 

means for sensing signal strength of the broadcast content signals; 

means for computing the distance between the transmitter and the receiver for 
identifying the distance from the vehicle to an origination position of the images, 
10 responsive to the means for sensing signal strength; and 

means for integrating the computed distance into the display presentation 
responsive to the computing. 

39. The system as in claim 1, further comprising: 

means for providing traffic flow direction for the user, and 
IS means for relaying selected ones of the respective local content signals in a 

backwards direction that is opposite of the traffic flow direction of the user. 

40. The system as in claim 14, wherein the system provides for display presentation 
on the display apparatus within at least one vehicle of surrounding traffic conditions for 
at least one of the plurality of sensors. 

20 41. The system as in claim 10, wherein the sensor apparatus further comprises means 
to variably position the sensor in at least one axis of orientation of pan, tilt, and roll. 

42. The system as in claim 40, wherein the variable positioning is further comprised 
of simultaneously variably positioning along a plurality of the axes. 

43. The system as in claim 40, wherein there are a plurality of vehicles each having 
25 relative forward and backward directions orientation within traffic flow, and 

wherein the system provides for display presentation within at least one of the 
vehicles of respective surrounding traffic conditions for at least one of the plurality of 
sensors in at least one of the forward direction and the backward direction. 

44. The system as in claim 22, wherein the system provides for visual 
30 communication. 
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45. The system as in claim 2 1 , wherein defmed content signals provide for emergency 
communications to a user in the vehicle. 

46. The system as in claim 45, wherein the presentation is at least one of audio, 
visual, and audiovisual. 

5 47. The system as in claim 45, wherein the defmed content signals provide for an 
emergency vehicle to provide communications for presentation to a plurality of separate 
ones of the vehicle. 

48. The system as in claim 1, further comprising an emergency vehicle 
communications subsystem providing for communications regarding the location of an 

10 emergency vehicle in proximity to the display apparatus for use by a user, to provide 
content signals to the receiver to provide notification to the user regarding the emergency 
vehicle. 

49. The system as in claim 22, further comprising an emergency vehicle 
communications subsystem providing for priority communications from an emergency 

15 vehicle to provide notification regarding the emergency vehicle to the plurality of 
vehicles. 

50. The system as in claim 22, wherein a uniform protocol is provided for 
compatibility in message structure between the plurality of vehicles. 

5 1 . The system as in claim 1, further comprising: 

20 means for providing a plurality of display modes; 

means for automatically cycling between selected ones of the plurality of display 

modes. 

52. The system as in claim 1, wherein external content signals provide for 
presentation of emergency vehicle trafftc conditions for presentation as at least one of an 

25 audio, a visual, and an audiovisual presentation. 

53. The system as in claim 52, wherein there are two levels for the content signals, a 
normal level and a priority level which is preemptive over the normal level for local 
display on the display apparatus, wherein the emergency vehicle traffic conditions are 
communicated at the priority level. 

30 54. The system as in claim 1, further comprising: 

a position detection subsystem providing position location data for a vehicle; 
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wherein the processing subsystem is responsive to the position location data to 
selectively process and format the content signals to provide a display presentation 
relative to the location of the vehicle. 

55. The system as in claim 1, wherein the display presentation is one of an audio, a 
S video, and an audiovisual presentation. 

56. The system as in claim 54, wherein the position detection subsystem utilizes a 
Global Positioning System. 

57. The system as in claim 25, wherein the associated position data is global 
positioning data. 

10 58. The method as in claim 22, further comprising: 

means for communicating emergency vehicle data as the content data to the 
plurality of vehicles which each provide means for generating a local presentation 
responsive thereto. 

59. The method as in claim 58, wherein the local presentation is one of an audio, a 
IS video, and an audiovisual presentation. 

60. The system as in claim 1, wherein the presentation is a display of a map 
indicating relative locations of the display apparatus of a user and known traffic 
conditions. 

61. The system as in claim 60, wherein the presentation is a display of a directional 
20 indication superimposed on the display of the road map indicating a way for the user to 

bypass congestion otherwise indicated in surrounding traffic conditions. 

62. The system as in claim 1, wherein the presentation is provided on a display 
internal to a vehicle. 

63. The system as in claim 1, wherein the presentation is provided on a display 
25 external to a vehicle. 

64. The system as in claim 1, wherein the content signals are further comprised of an 
associated data type, wherein the data type is at least one of video information, image 
information, congestion information, traffic pattern information, travel time information, 
road condition information, the vehicle's location, weather condition information, an 

30 emergency warning, construction alerts, a warning of locations of road hazards, a warning 
of disabled vehicles, a warning of slow moving vehicle locations. 



NSDOCID: <WO ^0143104A1_L> 



wo 01/43104 



PCT/USOO/33433 



-60- 

65. The system as in claim 64, wherein there are a plurality of images to select for 
display responsive to the content signals; 

wherein each of the content signals includes a source identification; 
wherein selection of selected ones of the plurality of images for display is 
S responsive to the source identification and the data type for the respective ones of the 
content signals. 

66. The system as in claim 65, wherein the content signals are processed for selection 
based on proximity to a defined reference point. 

67. The system as in claim 66, wherein the display apparatus is within a vehicle 
10 having an associated location; 

wherein the defined reference point is selected by the processing subsystem 
responsive to the associated location. 

68. The system as in claim 67, wherein the associated location is determined 
responsive to a Global Positioning System. 

IS 69. The system as in claim 1 , wherein there are a plurality of images to select for 

display responses to the broadcast content signals; 

wherein selection of selected ones of the plurality of images for display provides 

for display of at least one of traffic conditions for a forward direction, traffic conditions 

for a backrward direction, traffic conditions for along a planned route, computed travel 
20 times, congestion information, road hazard information, adverse weather condition 

information, and emergency vehicle information. 
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